moe_w8a8_grouped_gemm

Function moe_w8a8_grouped_gemm 

Source
pub fn moe_w8a8_grouped_gemm(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    a_fp8: DevicePtr,
    a_scale: DevicePtr,
    weight_ptrs: DevicePtr,
    scale_ptrs: DevicePtr,
    output: DevicePtr,
    expert_offsets: DevicePtr,
    sorted_token_ids: DevicePtr,
    num_experts: u32,
    n: u32,
    k: u32,
    max_m_tiles: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: W8A8 grouped MoE GEMM on a dense grid (ceil(N/64), max_m_tiles, num_experts). a_fp8 comes from per_token_group_quant_fp8; a_scale ([total_tokens, K/128] FP32) and the per-block b_scale (scale_ptrs[e] → [N/128, K/128] FP32) are applied in an FP32 epilogue per 128-element K-group. weight_ptrs[e] points at [N, K] FP8; output is [total_expanded, N] BF16; sorted_token_ids is [total_expanded] or NULL.