Crate metrale_cache

Crate metrale_cache 

Source
Expand description

2026-09-25: The paged KV block pool, host-side KV dequantization, the KV spill-file manager and the radix-tree prefix cache.

Owner: cache. Invariants: none beyond the types.

Modulesยง

kv_cache
2026-09-25: The paged KV cache: the KvCacheDtype storage formats, the per-layer block geometry in KvCacheConfig, and the PagedKvCache block pool with its per-block reference counts.
kv_dequant
2026-09-25: Host-side dequantization of paged KV cache blocks to BF16.
kv_spill
2026-09-25: Swap files for the --swap-space-gb spill of sequence state.
radix_tree
2026-09-25: The radix-tree prefix cache (PrefixCache implementation): KV block reuse plus the SSM snapshot index.