marconi_min_tokens

Function marconi_min_tokens 

Source
pub fn marconi_min_tokens() -> usize
Expand description

2026-09-25: Minimum snapshot depth, in matched tokens, for restoring an SSM snapshot; a shallower snapshot is not restored. 0 removes the floor.

A restore skips recomputing the matched prefix, so the drafter’s prompt prefill does not run, and a request with no earlier turn in its session has nothing to carry. Measured 2026-07-28 at C=1, warm against caching off: 99 matched tokens -9.7%, 219 +9.8%, 349 +7.5%, 629 +10.6%.

A serve sets the value from --marconi-min-tokens through set_marconi_min_tokens before building the model, so METRALE_MARCONI_MIN_TOKENS is used only when nothing set it first.