pub fn kquant_mmq_gemm(
gpu: &dyn GpuBackend,
kernel_nc: KernelHandle,
kernel_wc: KernelHandle,
a_q8: DevicePtr,
w_blocks: DevicePtr,
out_bf16: DevicePtr,
m: u32,
n: u32,
k: u32,
smem: u32,
stream: u64,
) -> Result<()>Expand description
2026-09-25: Prefill GEMM on raw blocks through the vendored MMQ engine:
out[m][n] (bf16) = A_q8[m][k] x W[n][k]. a_q8 must be in the type’s MMQ q8_1 layout
(D2S6 for Q2_K, D4 for Q3_K), and smem is the matching *_MMQ_SMEM. kernel_nc serves
n % 128 == 0, and kernel_wc (bounds-checked) any other n. The kernel’s arguments are
(x = weights, y = activations, dst, nrows_x = n, ncols_dst = m, ncols_x = k, stride_row_x = k / 256, ncols_y = m, stride_col_dst = n).