pub struct PromptLookupSeq { /* private fields */ }Expand description
2026-10-02: One sequence’s prompt-lookup state: its index, its copy window, and the length of the copy now awaiting verification (0 when the pending drafts are the model drafter’s).
Implementations§
Source§impl PromptLookupSeq
impl PromptLookupSeq
Sourcepub fn new(cfg: &PromptLookupConfig) -> Self
pub fn new(cfg: &PromptLookupConfig) -> Self
2026-10-02: Fresh state. The window starts at WINDOW_START (or the
ceiling, if lower), so an untested sequence’s first copies are short and
a wrong one costs few verify rows.
Sourcepub fn propose(&mut self, history: &[u32], max_len: usize) -> Option<Vec<u32>>
pub fn propose(&mut self, history: &[u32], max_len: usize) -> Option<Vec<u32>>
2026-10-02: Proposes a copy of at most max_len tokens for history
(prompt, generated tokens and the token about to be verified), and marks
it in flight. None (and nothing in flight) when there is no match or
max_len is 0. The caller bounds max_len, normally by
window.
No copy is proposed while a miss backoff is cooling down (each call
counts one round), or when the match spans fewer than min_match
tokens.
Sourcepub fn propose_with(
&mut self,
history: &[u32],
max_len: usize,
shared: impl FnOnce(&[u32], usize) -> Option<Vec<u32>>,
) -> Option<Vec<u32>>
pub fn propose_with( &mut self, history: &[u32], max_len: usize, shared: impl FnOnce(&[u32], usize) -> Option<Vec<u32>>, ) -> Option<Vec<u32>>
2026-10-04: propose, but when the sequence’s own
history has no match, asks shared (the cross-request cache) for a
copy of at most max_len tokens. The own history is asked first; the
cooldown applies to both; a shared copy is settled like an own one.
2026-10-04: The in-flight copy came from the cross-request cache.
Sourcepub fn in_flight(&self) -> usize
pub fn in_flight(&self) -> usize
2026-10-02: Length of the copy awaiting verification; 0 when none.
Sourcepub fn settle(&mut self, accepted: usize) -> usize
pub fn settle(&mut self, accepted: usize) -> usize
2026-10-02: Closes the in-flight copy: accepted of its tokens matched.
Returns the copy’s length (0 when none was in flight, which changes
nothing).
A copy that matched nothing starts (or doubles) a backoff of up to
miss_backoff rounds; any accepted token clears it.
Trait Implementations§
Source§impl Clone for PromptLookupSeq
impl Clone for PromptLookupSeq
Source§fn clone(&self) -> PromptLookupSeq
fn clone(&self) -> PromptLookupSeq
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more