per_token_group_quant_fp8

Function per_token_group_quant_fp8 

Source
pub fn per_token_group_quant_fp8(
    gpu: &dyn GpuBackend,
    quant: Fp8ActQuant,
    input_bf16: DevicePtr,
    output_fp8: DevicePtr,
    a_scale: DevicePtr,
    m: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: Per-token FP8 activation quantization over 128-element K-groups: A_fp8 [M, K] FP8 E4M3 and a_scale [M, K/128] FP32.

Fp8ActQuant::pick chooses the shared kernel or the Hopper twin for this width and returns the kernel and its grid together; both run 128-thread blocks. This is the only caller of fp8_quant_log, which logs the choice once per branch.