load_mtp

metrale_model_layers::weight_map::loaders_moe

Function load_mtp 

Source 
pub fn load_mtp(
    store: &WeightStore,
    num_experts: usize,
    gpu: &dyn GpuBackend,
    variant: Nvfp4Variant,
) -> Result<MtpWeights>
Expand description

2026-09-25: Load the MTP head (mtp.* keys) as BF16 DenseWeights.

Projections go through dense_auto (FP8 and packed NVFP4 are dequantized), except under Bf16Raw, which hands out the store pointers. Two FFN shapes:

  • MoE: router mtp.layers.0.mlp.gate.weight, per-expert or stacked expert tensors, and a shared expert.
  • Dense: mtp.layers.0.mlp.{gate,up,down}_proj.weight with no .gate.weight router, returned in dense_ffn.