w8a16_gemv_row_tiered

Function w8a16_gemv_row_tiered 

Source
pub fn w8a16_gemv_row_tiered(
    gpu: &dyn GpuBackend,
    gemv_kernel: KernelHandle,
    m32_kernel: KernelHandle,
    input: DevicePtr,
    weight: DevicePtr,
    block_scale: DevicePtr,
    output: DevicePtr,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-27: One row’s block-scaled W8A16 projection [1, k] x [n, k]^T for a single-row decode site: under RowTiers::Canonical, when m32_kernel is linked and k is whole 128-wide scale blocks, the 32-row tile (the order every other row count takes under that policy); otherwise the scalar w8a16_gemv.