Module kv_cache 

2026-09-25: The paged KV cache: the KvCacheDtype storage formats, the per-layer block geometry in KvCacheConfig, and the PagedKvCache block pool with its per-block reference counts.

Owner: cache. Invariants:

• Until release, every attention layer i < num_layers has its own K pool and V pool of num_blocks blocks, sized by that layer’s K-side and V-side block bytes.
• A block index is pushed back onto the free list only when a decrement (dec_ref, return_evicted_block) takes its count from 1 to 0. A decrement at 0 never pushes: it is logged, and dec_ref also panics in a debug build.

Structs

KvCacheConfig

2026-09-25: Geometry and formats of the paged KV cache.

PagedKvCache

2026-09-25: Paged KV cache across all attention layers: the pools and the block free list with per-block reference counts.

Enums

KvCacheDtype

2026-09-25: KV cache storage format. A symmetric variant uses one format for K and V; a *K*V variant names the K format, then the V format (kv_pair).