quantize_to_fp8

Function quantize_to_fp8 

Source
pub fn quantize_to_fp8(
    bf16_weight: &DenseWeight,
    n: usize,
    k: usize,
    gpu: &dyn GpuBackend,
    quantize_kernel: KernelHandle,
    stream: u64,
) -> Result<Fp8DenseWeight>
Expand description

2026-09-25: Quantize a BF16 dense weight [n, k] to FP8 E4M3 (n*k bytes) with one FP32 scale per row. It synchronizes stream, so it belongs at load time, not in a forward pass.