q2_0_mmq_gemm

Function q2_0_mmq_gemm 

Source
pub fn q2_0_mmq_gemm(
    gpu: &dyn GpuBackend,
    kernel_nc: KernelHandle,
    kernel_wc: KernelHandle,
    a_q8: DevicePtr,
    w_q2_0: DevicePtr,
    out_bf16: DevicePtr,
    m: u32,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: C[m, n] = A[m, k] x W[n, k] in BF16. a_q8 is the DS4 q8_1 activation from super::quantize_act_q8_1; w_q2_0 is the packed weight as loaded. The tile is 128 x 128 like the Q4_K GEMM, and Q2_0’s x tile row (MMQ_MMA_TILE_X_K_Q8_0) is the same 76 ints as Q4_K’s, so it reuses Q4K_MMQ_SMEM. kernel_wc is used when n is not a multiple of 128.