load_fp8_block_scaled_as_fp8weight

Function load_fp8_block_scaled_as_fp8weight 

Source
pub fn load_fp8_block_scaled_as_fp8weight(
    store: &WeightStore,
    prefix: &str,
    gpu: &dyn GpuBackend,
) -> Result<Fp8Weight>
Expand description

2026-09-25: Load an FP8 E4M3 block-scaled checkpoint weight as a native Fp8Weight.

{prefix}.weight must be a 2D FP8E4M3 tensor [N, K]. The block scale is read from the first of these that exists: {prefix}.weight_scale_inv, a 2D {prefix}.weight_scale, or a 2D F8_E8M0 {prefix}.scale. The scale must be BF16, FP32 or F8_E8M0, and it is widened once into an FP32 device buffer. When none exists, a scalar {prefix}.weight_scale is repeated over a [ceil(N/128), ceil(K/128)] FP32 grid. w8a16_gemv reads this buffer as const float* and applies block_scale[n/128, k/128] to each weight.