pub fn w8a16_gemv_batch4_strided(
gpu: &dyn GpuBackend,
kernel: KernelHandle,
input: DevicePtr,
weight: DevicePtr,
block_scale: DevicePtr,
output: DevicePtr,
m: u32,
n: u32,
k: u32,
a_row_stride: u32,
c_row_stride: u32,
stream: u64,
) -> Result<()>Expand description
2026-09-25: Strided variant of w8a16_gemv_batch4 (up to 4 rows). It
reads and writes rows at caller-given pitches, so one projection of the
multi-sequence decode QKV buffer [n, per_seq_qkv] is written for every
row in one launch.
Layout: input [M, a_row_stride] BF16, of which the first k elements
of each row are read; weight/block_scale as for w8a16_gemv
([N, K] FP8 E4M3 and [N/128, K/128] FP32); output [M, c_row_stride]
BF16, of which the first n elements of each row are written. Strides are
in elements. The launcher refuses pitches that do not cover k and n,
and an a_row_stride that is not a multiple of 8 (the kernel loads
activations as 16-byte uint4).
Per-row numerics as w8a16_gemv_batch4; native_fp8_qkv_batch_microtest
compares it with w8a16_gemv.
Kernel: w8a16_gemv_batch4_strided (module w8a16_gemv_batch4).