kquant_mmq_gemm

Function kquant_mmq_gemm 

Source
pub fn kquant_mmq_gemm(
    gpu: &dyn GpuBackend,
    kernel_nc: KernelHandle,
    kernel_wc: KernelHandle,
    a_q8: DevicePtr,
    w_blocks: DevicePtr,
    out_bf16: DevicePtr,
    m: u32,
    n: u32,
    k: u32,
    smem: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: Prefill GEMM on raw blocks through the vendored MMQ engine: out[m][n] (bf16) = A_q8[m][k] x W[n][k]. a_q8 must be in the type’s MMQ q8_1 layout (D2S6 for Q2_K, D4 for Q3_K), and smem is the matching *_MMQ_SMEM. kernel_nc serves n % 128 == 0, and kernel_wc (bounds-checked) any other n. The kernel’s arguments are (x = weights, y = activations, dst, nrows_x = n, ncols_dst = m, ncols_x = k, stride_row_x = k / 256, ncols_y = m, stride_col_dst = n).