Expand description
2026-09-25: The radix-tree prefix cache (PrefixCache implementation):
KV block reuse plus the SSM snapshot index.
Token sequences are chunked at block_size; each tree node stores one KV
block. A lookup walks block-aligned chunks and returns the cached block
indices, then asks the snapshot index for the deepest SSM snapshot within
the matched prefix.
Owner: cache. Invariants:
- The tree and the snapshot index are behind separate mutexes, and no
method holds both:
inneris released beforesnapshot_indexis taken. - One radix root per
adapter_id, so no lookup returns another adapter’s blocks.
Structs§
- Radix
Tree - 2026-09-25: The radix-tree prefix cache. SSM snapshots live in a separate
SsmSnapshotIndex, so evicting tree nodes does not remove them.