Module moe_cuda

Expand description

2026-09-25: Host launch of the Kimi K3 packed LatentMoE experts on the MXFP4 E8M0 pointer-table GEMM, PTRTABLE_E8M0 in PTX module MODULE.

A failed kernel lookup or a missing expert projection is an error: packed experts have no host fallback.

Owner: model-arch, Kimi K3. Invariants:

Every device buffer a launch allocates gets a free call before it returns, on success and on error; a failed free is ignored.

Structs§

E8M0_ENTRY
MODULE: 2026-09-25: The gb10 and b300 kimi-k3 KERNEL.toml files build this module from deepseek-v4-flash/nvfp4/moe_w4a16_grouped_gemm.cu ([sources] use, renamed by [modules]).
PTRTABLE_E8M0

launch_k3_latent_moe_experts: 2026-09-25: Computes what mix_routed_experts computes, sum_e mix_w[e] * w2(SiTU(w1 x, w3 x)) over ids, from the packed experts, with BF16 GEMM inputs and outputs. Empty ids returns zeros without a launch.
launch_k3_moe_e8m0_ptrtable: 2026-09-25: One grouped E8M0 GEMM, C[M, N] = A[M, K] @ B_packed[N, K/2], with one pointer-table entry per expert in packed. The launch covers one M tile per expert; gemm_rows gives every expert exactly one row.