Trait PrefixCache 

pub trait PrefixCache: Send + Sync {
    // Required methods
    fn lookup(
        &self,
        tokens: &[u32],
        block_size: usize,
        session_hash: u64,
        adapter_id: u64,
    ) -> PrefixMatch;
    fn insert(
        &self,
        tokens: &[u32],
        block_table: &[u32],
        disk_block_ids: &[u32],
        block_size: usize,
        matched_tokens: usize,
        adapter_id: u64,
    ) -> InsertAcquired;
    fn insert_with_snapshot(
        &self,
        tokens: &[u32],
        block_table: &[u32],
        disk_block_ids: &[u32],
        block_size: usize,
        snapshot_id: usize,
        session_hash: u64,
        matched_tokens: usize,
        adapter_id: u64,
    ) -> (Option<usize>, InsertAcquired);
    fn insert_intermediate_snapshot(
        &self,
        tokens: &[u32],
        block_table: &[u32],
        disk_block_ids: &[u32],
        block_size: usize,
        snapshot_id: usize,
        session_hash: u64,
        matched_tokens: usize,
        adapter_id: u64,
    ) -> Option<usize>;
    fn insert_tail_snapshot(
        &self,
        tokens: &[u32],
        snapshot_id: usize,
        session_hash: u64,
        adapter_id: u64,
    ) -> Vec<usize>;
    fn insert_tail_sibling_snapshot(
        &self,
        tokens: &[u32],
        snapshot_id: usize,
        session_hash: u64,
        adapter_id: u64,
    ) -> Option<usize>;
    fn release(&self, tokens: &[u32], block_size: usize, adapter_id: u64);
    fn release_matched(
        &self,
        tokens: &[u32],
        block_size: usize,
        matched_tokens: usize,
        adapter_id: u64,
    );
    fn evict(&self, num_blocks: usize) -> EvictedBlocks;
    fn evict_snapshot_lru(&self) -> Option<usize>;
    fn snapshot_count(&self) -> usize;
    fn stats(&self) -> (usize, usize);

    // Provided methods
    fn is_active(&self) -> bool { ... }
    fn peek_matched_tokens(
        &self,
        _tokens: &[u32],
        _block_size: usize,
        _adapter_id: u64,
    ) -> usize { ... }
    fn pin_prefix(
        &self,
        _tokens: &[u32],
        _block_size: usize,
        _adapter_id: u64,
    ) -> usize { ... }
    fn evict_snapshot_to_tier(&self, min_tokens: usize) -> Option<TierEvict> { ... }
    fn promote_snapshot(&self, key: u64, new_slot: usize) -> bool { ... }
    fn forget_snapshot_tier_key(&self, key: u64) -> bool { ... }
}

2026-09-26: A prefix cache. Every method takes &self, so an implementation keeps its state behind interior mutability.

Required Methods

fn lookup( &self, tokens: &[u32], block_size: usize, session_hash: u64, adapter_id: u64, ) -> PrefixMatch

2026-09-26: Look up tokens and return the cached KV blocks of the longest matching prefix, taking a reference on each full matched block so it survives eviction until PrefixCache::release.

session_hash scopes session-tail snapshots; adapter_id keys the cache, so only blocks and snapshots computed under the same adapter match (0 is the base model).

fn insert( &self, tokens: &[u32], block_table: &[u32], disk_block_ids: &[u32], block_size: usize, matched_tokens: usize, adapter_id: u64, ) -> InsertAcquired

2026-09-26: Insert a completed prefill’s blocks.

block_table[i] is the block for tokens [i*block_size .. (i+1)*block_size]. disk_block_ids is empty or parallel to block_table (--high-speed-swap). matched_tokens is the prefix the sequence already holds through lookup; the sequence gets a reference on every node past it, so that its release leaves the cache’s own reference in place. The caller applies the returned InsertAcquired.

fn insert_with_snapshot( &self, tokens: &[u32], block_table: &[u32], disk_block_ids: &[u32], block_size: usize, snapshot_id: usize, session_hash: u64, matched_tokens: usize, adapter_id: u64, ) -> (Option<usize>, InsertAcquired)

2026-09-26: insert, then register snapshot_id (a snapshot-pool slot) for the whole of tokens, tagged with session_hash. Returns the displaced resident slot for the caller to free, if any, and the insert’s InsertAcquired.

fn insert_intermediate_snapshot( &self, tokens: &[u32], block_table: &[u32], disk_block_ids: &[u32], block_size: usize, snapshot_id: usize, session_hash: u64, matched_tokens: usize, adapter_id: u64, ) -> Option<usize>

2026-09-26: Register snapshot_id for tokens, the sequence up to the snapshot point, tagged with session_hash. Returns the displaced resident slot for the caller to free, if any.

fn insert_tail_snapshot( &self, tokens: &[u32], snapshot_id: usize, session_hash: u64, adapter_id: u64, ) -> Vec<usize>

2026-09-26: Register the session’s tail snapshot in the index only. For a nonzero session_hash it replaces the session’s previous tail and tail sibling. Returns the displaced snapshot ids to free.

fn insert_tail_sibling_snapshot( &self, tokens: &[u32], snapshot_id: usize, session_hash: u64, adapter_id: u64, ) -> Option<usize>

2026-09-26: Register the tail’s sibling snapshot in the index. Call it after insert_tail_snapshot, which removes the session’s previous sibling. Returns a displaced snapshot id to free, if any.

fn release(&self, tokens: &[u32], block_size: usize, adapter_id: u64)

2026-09-26: Drop the sequence’s reference on each full block of tokens the cache holds, when the sequence finishes. adapter_id must be the one used at lookup and insert.

fn release_matched( &self, tokens: &[u32], block_size: usize, matched_tokens: usize, adapter_id: u64, )

2026-09-26: Drop the references one lookup took: the full blocks of the first matched_tokens tokens and no more, for a caller that rolls back a lookup before any sequence owns its blocks.

fn evict(&self, num_blocks: usize) -> EvictedBlocks

2026-09-26: Evict least-recently-used leaves that no sequence holds until at least num_blocks blocks are freed or none is left. A leaf’s partial-suffix block goes with it, so one more block than asked can come back. The caller applies the EvictedBlocks.

fn evict_snapshot_lru(&self) -> Option<usize>

2026-09-26: Evict one SSM snapshot from the index; returns its slot for the caller to free.

fn snapshot_count(&self) -> usize

2026-09-26: Number of SSM snapshots in the snapshot index.

fn stats(&self) -> (usize, usize)

2026-09-26: (entries, cached_blocks). RadixTree stores one block per entry and answers its node count for both.

Provided Methods

fn is_active(&self) -> bool

2026-09-26: Whether this cache stores blocks and holds references. NoPrefixCaching answers false, and callers then skip the cache’s bookkeeping.

fn peek_matched_tokens( &self, _tokens: &[u32], _block_size: usize, _adapter_id: u64, ) -> usize

2026-09-26: The number of tokens lookup would match, without taking references, touching LRU state or counting a hit or miss. Keyed by adapter_id like lookup. The default answers 0.

fn pin_prefix( &self, _tokens: &[u32], _block_size: usize, _adapter_id: u64, ) -> usize

2026-09-30: Take a reference on each full block of the prefix lookup would match, without the snapshot lookup or a hit or miss count, so eviction skips that prefix until PrefixCache::release_matched with the returned token count. The default pins nothing and answers 0.

fn evict_snapshot_to_tier(&self, min_tokens: usize) -> Option<TierEvict>

2026-09-26: Pick a resident snapshot to evict: one at least min_tokens deep is marked spilled and stays findable, a shallower one is removed (TierEvict). min_tokens == 0 always spills. None when nothing is resident; the default answers None.

fn promote_snapshot(&self, key: u64, new_slot: usize) -> bool

2026-09-26: After the caller faulted a spilled snapshot into new_slot, make its entry resident there. false if the key is unknown; the default answers false.

fn forget_snapshot_tier_key(&self, key: u64) -> bool

2026-09-26: Remove the spilled entry for key, for a caller whose stored bytes for it are gone, so later lookups stop returning it. A resident entry is left alone, since its slot is live. Returns whether an entry was removed; the default answers false.

Implementors

impl PrefixCache for NoPrefixCaching