requant_w_nvfp4_int8

Function requant_w_nvfp4_int8 

Source
pub fn requant_w_nvfp4_int8(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    w_packed: DevicePtr,
    w_e4m3: DevicePtr,
    scale2: f32,
    w_i8: DevicePtr,
    w_scale: DevicePtr,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: Requantize an NVFP4 weight (packed E2M1, per-16 E4M3 block scales, per-tensor scale2) to int8 with one F32 scale per 32 elements, for int8_gemm_faith2. Reads W_packed[N, K/2] and W_e4m3[N, K/16]; writes W_i8[N, K] (signed int8) and W_scale[N, K/32] (F32). The dense FFN’s ensure_int8_weight runs it on a weight’s first int8 prefill and keeps the result.