load_nemotron_moe

metrale_model_layers::weight_map::nemotron

Function load_nemotron_moe 

Source 
pub fn load_nemotron_moe(
    store: &WeightStore,
    layer: usize,
    num_experts: usize,
    gpu: &dyn GpuBackend,
    config: &ModelConfig,
    absmax_k: Option<KernelHandle>,
    quantize_k: Option<KernelHandle>,
    stream: u64,
    scratch: Option<DevicePtr>,
    layer_prefix: &str,
) -> Result<NemotronMoeWeights>
Expand description

2026-09-25: Load a Nemotron-H MoE mixer.

Expert projections that are not NVFP4 on disk (FP8 or BF16) are quantized to NVFP4 at load, except shared-expert projections kept as native FP8. That needs absmax_k and quantize_k (it panics without them). Expert FP8 dequants go into scratch when it is given. The LatentMoE projections stay BF16.