q2_0_gemv_batchm

Function q2_0_gemv_batchm 

Source
pub fn q2_0_gemv_batchm(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    input: DevicePtr,
    weight: &PackedQ2Weight,
    output: DevicePtr,
    m: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: Batched Q2_0 GEMV: C[M,N] = A[M,K] @ dequant(B)^T, A and C row-major BF16. Each weight block is read once and applied to all m rows.

m must be in 1..=8: the kernel keeps acc[MAX_M] with MAX_M = 8 and indexes it up to M. This launcher does not check it.