pub fn q2_0_gemv_vec_batchm(
gpu: &dyn GpuBackend,
kernel: KernelHandle,
input: DevicePtr,
weight: &PackedQ2Weight,
output: DevicePtr,
m: u32,
stream: u64,
) -> Result<()>Expand description
2026-09-25: Batched Q2_0 GEMV for any m: C[M,N] = A[M,K] @ dequant(B)^T, A and C
row-major BF16. Each weight word is read once per launch and applied to every staged row.
m is split into launches of at most Q2_BATCHM_MAX_M rows, with the input and output
pointers advanced by whole rows; m <= 8 is one launch, and m == 0 launches nothing.