pub struct MlaWeights {Show 38 fields
pub wq_a: DenseWeight,
pub wq_a_nvfp4: Option<QuantizedWeight>,
pub wq_a_fp8: Option<Fp8Weight>,
pub wq_b: DenseWeight,
pub wq_b_nvfp4: Option<QuantizedWeight>,
pub wq_b_fp8: Option<Fp8Weight>,
pub q_a_norm: DenseWeight,
pub wkv_a: DenseWeight,
pub wkv_a_nvfp4: Option<QuantizedWeight>,
pub wkv_a_fp8: Option<Fp8Weight>,
pub wkv_b: DenseWeight,
pub kv_a_norm: DenseWeight,
pub wkv_a_rope: DenseWeight,
pub wkv_a_merged: DenseWeight,
pub wo: DenseWeight,
pub wo_nvfp4: Option<QuantizedWeight>,
pub wo_a: DenseWeight,
pub wo_a_nvfp4: Option<QuantizedWeight>,
pub wo_a_fp8: Option<Fp8Weight>,
pub wo_b: DenseWeight,
pub wo_b_nvfp4: Option<QuantizedWeight>,
pub wo_b_fp8: Option<Fp8Weight>,
pub w_uk_t: DenseWeight,
pub w_uv: DenseWeight,
pub wq_b_rope: DenseWeight,
pub w_qk_absorbed: DenseWeight,
pub w_uk_block_diag: DenseWeight,
pub w_uv_block_diag: DenseWeight,
pub yarn_inv_freq: DevicePtr,
pub main_inv_freq: DevicePtr,
pub q_lora_rank: usize,
pub kv_lora_rank: usize,
pub o_lora_rank: usize,
pub nope: usize,
pub rope: usize,
pub v_dim: usize,
pub compressor: Option<CompressorWeights>,
pub attn_sink: DevicePtr,
}Expand description
2026-09-25: MLA (multi-head latent attention) weights. Q is projected in two
steps: input × wq_a → latent, the q_a_norm RMS norm, then
latent × wq_b → Q.
Fields§
§wq_a: DenseWeight§wq_a_nvfp4: Option<QuantizedWeight>§wq_a_fp8: Option<Fp8Weight>2026-09-25: Native block-scaled FP8 copy (E4M3 with 128x128 block
scales), read by the decode w8a16_gemv.
wq_b: DenseWeight§wq_b_nvfp4: Option<QuantizedWeight>§wq_b_fp8: Option<Fp8Weight>§q_a_norm: DenseWeight§wkv_a: DenseWeight§wkv_a_nvfp4: Option<QuantizedWeight>§wkv_a_fp8: Option<Fp8Weight>§wkv_b: DenseWeight§kv_a_norm: DenseWeight§wkv_a_rope: DenseWeight§wkv_a_merged: DenseWeight§wo: DenseWeight§wo_nvfp4: Option<QuantizedWeight>§wo_a: DenseWeight2026-09-25: Grouped low-rank O projection (wo_a then wo_b), used by
the decode and prefill paths when o_lora_rank > 0 instead of wo.
wo_a_nvfp4: Option<QuantizedWeight>§wo_a_fp8: Option<Fp8Weight>2026-09-25: Native block-scaled FP8 wo_a for the grouped decode
O-projection.
wo_b: DenseWeight§wo_b_nvfp4: Option<QuantizedWeight>§wo_b_fp8: Option<Fp8Weight>§w_uk_t: DenseWeight§w_uv: DenseWeight§wq_b_rope: DenseWeight§w_qk_absorbed: DenseWeight§w_uk_block_diag: DenseWeight§w_uv_block_diag: DenseWeight§yarn_inv_freq: DevicePtr§main_inv_freq: DevicePtr2026-09-25: Plain inverse frequencies without YaRN, for the Q/K RoPE on
layers without a compressor; layers with one (CSA/HCA) use
yarn_inv_freq.
q_lora_rank: usize§kv_lora_rank: usize§o_lora_rank: usize§nope: usize§rope: usize§v_dim: usize§compressor: Option<CompressorWeights>2026-09-25: DeepSeek-V4 compressed-attention compressor; None when the
layer’s compress_ratios entry is 0 (full attention).
attn_sink: DevicePtr2026-09-25: Per-head attention sink logits, FP32 (the loader converts a BF16 checkpoint tensor); NULL when the checkpoint has none for this layer.