Struct SharedTokenCache 

pub struct SharedTokenCache { /* private fields */ }

2026-10-04: The cross-request cache: scopes created on first insert, at most max_scopes, each of fixed size.

Implementations

impl SharedTokenCache

pub fn new(cfg: SharedLookupConfig) -> Result<Self, String>

2026-10-04: An empty cache; Err when a scope would be too small (SharedLookupConfig::scope_geometry).

pub fn key_len(&self) -> usize

2026-10-04: Tokens a match must span.

pub fn scope_count(&self) -> usize

2026-10-04: Scopes now held.

pub fn stored_tokens(&self, scope: &ScopeKey) -> u64

2026-10-04: Ring tokens written under scope so far (0 when absent): what a document’s stored, not skipped, tokens cost.

pub fn insert(&mut self, scope: ScopeKey, doc: &[u32])

2026-10-04: Adds one request’s tokens (prompt then output) under scope, creating the scope if needed. When max_scopes already exist the least recently used one is dropped first.

pub fn propose( &mut self, scope: &ScopeKey, history: &[u32], max_len: usize, ) -> Option<Vec<u32>>

2026-10-04: The continuation, at most max_len tokens, of the latest stored occurrence under scope of history’s last key_len tokens. None when the scope does not exist or holds no such occurrence.

Trait Implementations

impl Debug for SharedTokenCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.