dequant_q2_0_gn_to_bf16

Function dequant_q2_0_gn_to_bf16 

Source
pub fn dequant_q2_0_gn_to_bf16(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    blocks: DevicePtr,
    out: DevicePtr,
    n: u32,
    k: u32,
    group: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: Dequantize a packed Q2_0 weight [N, K] into the caller’s BF16 buffer [N, K] on stream, with no allocation and no host sync (kernel dequant_q2_0_gn_to_bf16 in kernels/gb10/common/dequant_gguf_bf16.cu). Packed-Q2 prefill uses it to feed a BF16 GEMM while the resident weight stays packed (dense_ffn.rs, qwen3_attention/prefill_weights.rs).

k must be a multiple of group. One CTA per block_q2_0: n * (k / group) blocks of 2 + group/4 bytes, each expanding to group BF16 values.