Module prefix_cache 

2026-09-26: The PrefixCache trait (KV block reuse across requests that share a prompt prefix, plus the SSM snapshot index) and the prefix-cache hit and miss counters.

Implementations: NoPrefixCaching (caching disabled) and the metrale-cache crate’s RadixTree.

Owner: telemetry. Invariants: none beyond the types.

Structs

EvictedBlocks

2026-09-26: What PrefixCache::evict removed.

InsertAcquired

2026-09-26: What an insert started or stopped storing, so the caller can take and release the matching references.

NoPrefixCaching

2026-09-26: No-op prefix cache: is_active is false, every lookup is empty, and every insert and eviction reports nothing.

PrefixMatch

2026-09-26: Result of looking up a token sequence in the prefix cache.

Enums

TierEvict

2026-09-26: Outcome of crate::prefix_cache::PrefixCache::evict_snapshot_to_tier.

Traits

PrefixCache

2026-09-26: A prefix cache. Every method takes &self, so an implementation keeps its state behind interior mutability.

Functions

cache_hit_count

cache_hit_tokens_total

cache_miss_count

record_cache_hit

record_cache_miss