Module qwen3_attention 

2026-09-25: The full-attention transformer layer, Qwen3AttentionLayer: its construction, kernel requirements, decode and prefill attention, and the call into the layer’s FFN.

Owner: model-layers (attention). Invariants: none beyond the types.

Re-exports

pub use innerq_driver::InnerQDriver;

pub use types::Qwen3AttentionLayer;

Modules

helpers

2026-09-25: Qwen3AttentionLayer setters that model loaders call after construction, the YaRN mscale helper, and small per-layer helpers (apply_layer_scalar, effective_attn_scale, nvfp4_decode_gemv, qsa_seq_state).

innerq_driver

2026-09-25: Host-side driver for InnerQ, the per-channel K equalization of the WHT-rotated (turbo) KV dtypes.

types

2026-09-25: The Qwen3AttentionLayer struct: its weights, per-layer settings and resolved kernel handles.

Structs

CompressorWeights

2026-09-25: DeepSeek-V4 compressed-attention compressor weights, one per layer with a non-zero compress_ratios entry. CSA layers use a 2×ratio overlapping window, HCA layers a single window.

Fp8TwinSet

2026-09-25: Which of the four attention projections get an FP8 [K, N] transposed twin built by Qwen3AttentionLayer::transpose_fp8_for_prefill.

HcHeadWeights

2026-09-25: Model-level HC head parameters: the final stream collapse before the LM head. Only the last model layer uses them.

HcLowRank

2026-09-25: The low-rank hyper-connection parameters for one site (qwen4_exp).

HcSiteWeights

2026-09-25: Hyper-connection (mHC) parameters for one site (attention or FFN). The Sinkhorn fields are FP32; with lowrank set they are NULL.

HcWeights

MlaWeights

2026-09-25: MLA (multi-head latent attention) weights. Q is projected in two steps: input × wq_a → latent, the q_a_norm RMS norm, then latent × wq_b → Q.

Constants

W8A8_PREFILL_KERNELS

2026-09-25: The two (module, function) pairs the W8A8 block-scaled prefill arm needs: the activation quantizer and fp8_gemm_t_blockscaled.

Statics

ATTN_PHASE_US

2026-09-25: Per-phase host time in µs for the prefill attention path, written only by prefill/cache_skip.rs: 0 = the Q/K/V projections, 1 = from the projections to the BR=64 attention call, 2 = that attention call. No code writes slot 3. Summed across layers; the layer loop takes them once per prefill when METRALE_PREFILL_HOST_TIMING=1.

FFN_HOST_US

2026-09-25: Host time in µs spent in the FFN of prefill attention layers, added when METRALE_PREFILL_HOST_TIMING=1. Summed across layers; the layer loop takes it once per prefill and reports the attention remainder as the in-layer time minus this.

Functions

add_attn_phase_us

add_ffn_host_us

take_attn_phase_us

take_ffn_host_us

validate_required_kv_kernels

2026-09-25: Startup check for one --kv-cache-dtype: resolves every kernel the dtype’s dispatch needs (the reshape/decode pair, the chunked-prefill kernel and, for a WHT-rotated side, the WHT bookends) and errors with the full missing list. The server calls it before build_model. See kernel_requirements.rs.

w8a8_prefill_kernels_loaded

2026-09-25: Whether both W8A8_PREFILL_KERNELS resolve on this backend (through try_kernel), without a constructed layer.