quantize_bf16_to_nvfp4

Function quantize_bf16_to_nvfp4 

Source
pub fn quantize_bf16_to_nvfp4(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    input: DevicePtr,
    packed_out: DevicePtr,
    scale_out: DevicePtr,
    m: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: Quantize a BF16 [M, K] matrix to NVFP4 with scale2 = 1.0: packed E2M1 [M, K/2] and one E4M3 scale per 16 elements [M, K/16]. One block per row; the kernel’s N parameter is the row count. Produces the W4A4 prefill activations.