pub fn q4k_mmq_gemm(
gpu: &dyn GpuBackend,
kernel_nc: KernelHandle,
kernel_wc: KernelHandle,
a_q8: DevicePtr,
w_q4k: DevicePtr,
out_bf16: DevicePtr,
m: u32,
n: u32,
k: u32,
stream: u64,
) -> Result<()>Expand description
2026-09-25: C[m, n] = A[m, k] x W[n, k] in BF16, with a_q8 in q8_1 and
w_q4k in block_q4_K, on a 128 x 128 tile. kernel_wc is used when n
is not a multiple of 128.