pub fn decode_w8a8_quant_act(
gpu: &dyn GpuBackend,
scratch: &DecodeW8a8Scratch,
act_bf16: DevicePtr,
rows: u32,
k: u32,
stream: u64,
) -> Result<()>Expand description
2026-09-25: Quantize BF16 act[rows, k] into the scratch: FP8 E4M3 bytes
and one FP32 scale per token and 128 of K, with the pad rows
rows..ceil16(rows) zeroed, and the scales transposed for cuBLASLt when
the layout is K-major.
Separate from the GEMM so projections over one activation share one quantization; the attention Q/K/V do.
The pad rows’ FP8 bytes are zeroed as well as their scales: a zero scale
alone is not enough, because NaN * 0.0 is NaN.