q2_0_gemv_vec

Function q2_0_gemv_vec 

Source
pub fn q2_0_gemv_vec(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    input: DevicePtr,
    weight: &PackedQ2Weight,
    output: DevicePtr,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: Q2_0 GEMV for M=1 decode: C[1,N] = A[1,K] @ dequant(B)^T. Each lane reads one uint32 of 16 codes, and the CTA stages the activation tile in shared memory. A and C are BF16; B is the raw block_q2_0 buffer; each weight is (code-1)*d.