dequant_fp8_blockscaled_to_bf16

metrale_model_layers::weight_map::quant_helpers

Function dequant_fp8_blockscaled_to_bf16 

Source 
pub fn dequant_fp8_blockscaled_to_bf16(
    store: &WeightStore,
    prefix: &str,
    gpu: &dyn GpuBackend,
) -> Result<DenseWeight>
Expand description

2026-09-25: Dequantize a block-scaled FP8 E4M3 weight {prefix}.weight [N, K] into a new BF16 device buffer: bf16[i,j] = fp8[i,j] * scale[i/block_n, j/block_k], with the block sizes inferred from the scale’s shape.

A weight_scale_inv, or else a 2-D weight_scale, must be BF16 or FP32 and is applied on the device by dequant_fp8_blockscaled_bf16. Otherwise a 1-D weight_scale or a .scale (FP32, BF16 or E8M0) is applied on the host. Having none of them is an error.