q2_dequant_scratch_bytes

Function q2_dequant_scratch_bytes 

Source
pub fn q2_dequant_scratch_bytes(config: &ModelConfig) -> usize
Expand description

2026-09-25: Bytes of the keep-packed Q2_0 dequant scratch: hidden_size times the widest of intermediate, QKVZ, Mamba-2 in_proj, the q projection and 2 · kv_heads · head_dim, in BF16. This covers a weight [N, K] with hidden_size as one dimension and the other at most that width; the dequant callers debug-assert N * K * 2 against the allocated size. It does not depend on the batch.