quantize_to_nvfp4

Function quantize_to_nvfp4 

Source
pub fn quantize_to_nvfp4(
    bf16_weight: &DenseWeight,
    n: usize,
    k: usize,
    gpu: &dyn GpuBackend,
    absmax_kernel: KernelHandle,
    quantize_kernel: KernelHandle,
    stream: u64,
) -> Result<QuantizedWeight>
Expand description

2026-09-25: Quantize a BF16 dense [n, k] weight to NVFP4 on the GPU.

Two kernels: absmax_kernel finds the global absolute max, then quantize_kernel writes packed E2M1 (n*k/2 bytes) and one scale byte per 16 weights. The global scale is global_max / (6 * 448), or 1.0 for an all-zero weight. It synchronizes stream after each kernel, so it belongs at load time, not in a forward pass.