Module kv_dequant 

2026-09-25: Host-side dequantization of paged KV cache blocks to BF16.

The --high-speed-swap offload in metrale-model-layers (qwen3_attention/decode/high_speed_swap.rs) uses it to turn quantized K and V blocks into the BF16 it offloads. The layouts follow the kernels (kernels/gb10/common/):

Quant	data bytes/elem	scale	LUT	kernel
FP8	1 (E4M3)	per tensor	e4m3_lut	reshape_and_cache_flash_fp8 (reshape_and_cache.cu)
NVFP4	0.5 (4-bit)	FP8 per group	NVFP4_E2M1_LUT	paged_decode_attn_nvfp4.cu
Turbo4	0.5 (4-bit)	FP8 per group	TURBO4_LUT (16 levels)	paged_decode_attn_turbo4.cu
Turbo3	0.375 (3-bit)	FP8 per group	TURBO3_LUT (8 levels)	paged_decode_attn_turbo3.cu
Turbo8	1 (E4M3)	BF16 per group	e4m3_lut	paged_decode_attn_turbo8.cu

A group is NVFP4_GROUP_SIZE (16) elements. Group scales sit in a section after the block’s data section.

Owner: cache. Invariants: none beyond the types.

Constants

NVFP4_E2M1_LUT

2026-09-25: E2M1 4-bit codebook (NVFP4), as in kernels/gb10/common/paged_decode_attn_nvfp4.cu.

NVFP4_GROUP_SIZE

2026-09-25: Elements per scale group in every grouped layout; the NVFP4_GROUP_SIZE of kernels/gb10/common/paged_decode_attn_nvfp4.cu.

TURBO3_LUT

2026-09-25: Turbo3 8-level Lloyd-Max codebook, as in kernels/gb10/common/paged_decode_attn_turbo3.cu.

TURBO4_LUT

2026-09-25: Turbo4 16-level Lloyd-Max codebook, as in kernels/gb10/common/paged_decode_attn_turbo4.cu.

Functions

dequant_4bit_block_to_bf16

2026-09-25: A 4-bit block (NVFP4 or Turbo4, chosen by lut) to BF16.

dequant_fp8_to_bf16

2026-09-25: FP8 E4M3 bytes to BF16, times a per-tensor scale.

dequant_turbo3_block_to_bf16

2026-09-25: A Turbo3 block to BF16.

dequant_turbo8_block_to_bf16

2026-09-25: A Turbo8 block (FP8 E4M3 data, BF16 group scales) to BF16.

e4m3_lut

2026-09-25: The E4M3 table.