pub fn moe_w8a8_grouped_gemm(
gpu: &dyn GpuBackend,
kernel: KernelHandle,
a_fp8: DevicePtr,
a_scale: DevicePtr,
weight_ptrs: DevicePtr,
scale_ptrs: DevicePtr,
output: DevicePtr,
expert_offsets: DevicePtr,
sorted_token_ids: DevicePtr,
num_experts: u32,
n: u32,
k: u32,
max_m_tiles: u32,
stream: u64,
) -> Result<()>Expand description
2026-09-25: W8A8 grouped MoE GEMM on a dense grid
(ceil(N/64), max_m_tiles, num_experts). a_fp8 comes from
per_token_group_quant_fp8; a_scale ([total_tokens, K/128] FP32) and
the per-block b_scale (scale_ptrs[e] → [N/128, K/128] FP32) are
applied in an FP32 epilogue per 128-element K-group. weight_ptrs[e]
points at [N, K] FP8; output is [total_expanded, N] BF16;
sorted_token_ids is [total_expanded] or NULL.