nvfp4_grouped_gate_up

Function nvfp4_grouped_gate_up 

Source
pub fn nvfp4_grouped_gate_up(
    a: u64,
    gate_packed_ptrs: &[u64],
    gate_scale_ptrs: &[u64],
    gate_scale2_vals: &[f32],
    up_packed_ptrs: &[u64],
    up_scale_ptrs: &[u64],
    up_scale2_vals: &[f32],
    c_gate: u64,
    c_up: u64,
    expert_offsets: &[i32],
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: Per-expert NVFP4 gate and up GEMMs: for each expert with rows, the C side calls metrale_cutlass_nvfp4_gemm_bf16_act_weight_t once for gate and once for up, so the result matches nvfp4_gemm_bf16_act_weight_t on each expert’s rows.

a is bf16 [M_total, K]; expert e owns rows [expert_offsets[e], expert_offsets[e+1]). *_packed_ptrs and *_scale_ptrs hold one device pointer per expert, in the pack_bf16_weight_to_nvfp4_t layout ([N,K/2] and [K/16,N]). The *_scale2_vals and expert_offsets slices are host arrays.