q4k_mmq_gemm

Function q4k_mmq_gemm 

Source
pub fn q4k_mmq_gemm(
    gpu: &dyn GpuBackend,
    kernel_nc: KernelHandle,
    kernel_wc: KernelHandle,
    a_q8: DevicePtr,
    w_q4k: DevicePtr,
    out_bf16: DevicePtr,
    m: u32,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: C[m, n] = A[m, k] x W[n, k] in BF16, with a_q8 in q8_1 and w_q4k in block_q4_K, on a 128 x 128 tile. kernel_wc is used when n is not a multiple of 128.