pub fn w8a16_gemv_row_tiered(
gpu: &dyn GpuBackend,
gemv_kernel: KernelHandle,
m32_kernel: KernelHandle,
input: DevicePtr,
weight: DevicePtr,
block_scale: DevicePtr,
output: DevicePtr,
n: u32,
k: u32,
stream: u64,
) -> Result<()>Expand description
2026-09-27: One row’s block-scaled W8A16 projection [1, k] x [n, k]^T for
a single-row decode site: under RowTiers::Canonical, when m32_kernel is
linked and k is whole 128-wide scale blocks, the 32-row tile (the order
every other row count takes under that policy); otherwise the scalar
w8a16_gemv.