Struct SequenceState 

pub struct SequenceState {

    pub tokens: Vec<u32>,
    pub block_table: Vec<u32>,
    pub seq_len: usize,
    pub layer_states: Vec<Box<dyn LayerState>>,
    pub proposer_state: Option<Box<dyn ProposerState>>,
    pub slot_idx: usize,
    pub marconi_skip_to: usize,
    pub marconi_exact_snap: Option<usize>,
    pub session_hash: u64,
    pub lookup_tenant: Option<u64>,
    pub mtp_capture_gen: u64,
    pub mtp_store_gen: u64,
    pub adapter_id: u64,
    pub chunked_prefill_meta: Option<ChunkedPrefillPageMetadata>,
    pub cached_prefix_tokens: usize,
    pub reused_prefix_tokens: usize,
    pub cached_prefix_blocks: usize,
    pub prefix_ref_tokens: Vec<u32>,
    pub prefix_lookup_applied: bool,
    pub prefix_lookup_skip: bool,
    pub tail_checkpoint_tokens: Option<usize>,
    pub kv_valid_tokens: usize,
    pub last_decode_ckpt_block: usize,
    pub prompt_len: usize,
    pub disk_block_ids: Vec<u32>,
    pub disk_last_offloaded_per_layer: Vec<u32>,
    pub collect_prompt_logprobs: Option<u8>,
    pub prompt_logprobs: Vec<PromptTokenLogprob>,
    pub adapter_slot: i32,
    pub acquired_adapter_slot: i32,
    pub src_lang_id: u32,
    pub tgt_lang_id: u32,
    pub num_beams: u32,
    pub length_penalty: f32,
    pub early_stopping: bool,
    /* private fields */
}

2026-09-25: One sequence’s state across prefill and decode.

Fields

tokens: Vec<u32>

2026-09-25: Prompt and generated token ids processed so far.

block_table: Vec<u32>

2026-09-25: Physical KV blocks of the paged cache, in logical order.

seq_len: usize

layer_states: Vec<Box<dyn LayerState>>

proposer_state: Option<Box<dyn ProposerState>>

2026-09-25: The draft proposer’s per-sequence state; None without a proposer.

slot_idx: usize

2026-09-25: Pool slot of this sequence, taken from ssm_slot at alloc_sequence. compact_sequence moves it, and detach_slot_for_reuse sets it to usize::MAX.

marconi_skip_to: usize

2026-09-25: Prompt position covered by the restored SSM snapshot and cached KV, set by the chunk-0 prefix lookup and returned again when that lookup is replayed.

marconi_exact_snap: Option<usize>

2026-09-25: Snapshot slot when the whole prompt matched a snapshot. The last prompt token is then re-run for logits, which advances the SSM state a second time, so finalize_last restores the state from this snapshot again and takes the first token’s hidden row from it. None otherwise.

session_hash: u64

2026-09-25: Session identity, set by the scheduler before prefill. Saved SSM snapshots are tagged with it and a snapshot is restored only for a matching tag (SsmSnapshotPool::session_matches); 0 skips the check.

lookup_tenant: Option<u64>

2026-10-04: Tenant for the scheduler’s cross-request prompt-lookup cache, set by the scheduler before prefill and carried across preemption and swap. None keeps the sequence out of that cache (neither read nor written).

mtp_capture_gen: u64

2026-09-25: This sequence’s claim on the single shared whole-prompt hidden capture (mtp_prefill_hidden). try_mtp_prefill_capture sets it to a new capture generation when it writes this sequence’s rows from row 0. ensure_drafter_context prefills the drafter only while it equals the model’s current generation, since another sequence’s chunk 0 may have restarted the capture in between. 0 means it never owned a capture.

mtp_store_gen: u64

2026-09-25: This sequence’s ticket for the shared hidden-row interval (mtp_store_range), drawn in alloc_sequence from the model’s counter, and separate from mtp_capture_gen, which is set only by a capture that starts at row 0. 0 means it was not drawn in alloc_sequence, and matches no owner.

adapter_id: u64

2026-09-25: Prefix-cache namespace of the sequence’s LoRA adapter; 0 is the base model. Each id has its own radix root, so adapters never reuse each other’s blocks. The scheduler sets it from Model::adapter_id_for.

chunked_prefill_meta: Option<ChunkedPrefillPageMetadata>

2026-09-25: Paged metadata for chunked prefill, allocated by the first chunk that needs it.

cached_prefix_tokens: usize

2026-09-25: Prompt tokens matched by the chunk-0 prefix-cache lookup. The block-ref accounting reads it (the release in free_sequence, the prefill-end cache insert). It is not what clients are told: a match can be found and then recomputed (reused_prefix_tokens).

reused_prefix_tokens: usize

2026-09-25: Prompt tokens whose KV this request read from the prefix cache instead of recomputing (prefix_reuse::reused_prefix_tokens). The scheduler reports it as usage.prompt_tokens_details.cached_tokens.

cached_prefix_blocks: usize

2026-09-25: Number of block_table entries that came from the prefix-cache lookup (matched_blocks.len()); 0 without a hit.

prefix_ref_tokens: Vec<u32>

2026-09-25: The matched prefix tokens (tokens[..cached_prefix_tokens]), stored at lookup. When a prefill matched a prefix and then failed before tokens was filled, free_sequence releases the lookup’s radix refs over these instead. Empty when nothing matched.

prefix_lookup_applied: bool

2026-09-25: Whether the chunk-0 prefix lookup has run for this sequence.

The lookup is not idempotent: it takes radix refs, inc_refs each matched block and pushes it onto block_table, all before the suffix is allocated. When that allocation fails, the scheduler’s preempt-and-retry re-enters chunk 0; with this flag set the lookup replays its first result instead of pushing the blocks and taking the refs again.

prefix_lookup_skip: bool

2026-09-25: The skip result of the chunk-0 lookup, returned again on a replay.

tail_checkpoint_tokens: Option<usize>

2026-09-25: Token count of an SSM checkpoint near the prompt’s end that the pool holds for this sequence: saved during this prefill, or the one a warm prefill restored from. Cleared at chunk 0; finalize_last reads it to decide whether the prompt-end leaf gets its own snapshot (prefill_b::exact_leaf).

kv_valid_tokens: usize

2026-09-25: Length of the prefix, from position 0, whose paged KV this sequence has written or validly reused, updated per chunk by prefill_b_proc_range. The prefill-end cache insert and checkpoint save cover at most kv_valid_tokens / block_size complete blocks, so a block whose KV was never written is not cached.

last_decode_ckpt_block: usize

2026-09-25: Block index (len / block_size) of the latest decode-time SSM checkpoint, so a boundary is not saved twice. Prefill sets it to the prompt’s block count.

prompt_len: usize

2026-09-25: Prompt length, set by prefill. cache_sequence passes it to the prefix-cache insert as matched_tokens, so the insert takes refs only for the generated tokens.

disk_block_ids: Vec<u32>

2026-09-25: --high-speed-swap disk block ids, one per logical block the sequence has had, in order. When the HBM window is at its cap, the oldest block_table entry is freed and its disk id stays (ensure_blocks_through_decode), so block_table holds the newest blocks and hss_window_start() is the logical index of block_table[0]. Empty without --high-speed-swap.

disk_last_offloaded_per_layer: Vec<u32>

2026-09-25: For each attention layer, how many disk_block_ids entries it has offloaded. A block leaves the HBM window only when every layer has offloaded it (check_safe_to_evict). TransformerModel::alloc_sequence sizes it to the attention-layer count whether or not --high-speed-swap is on.

collect_prompt_logprobs: Option<u8>

2026-09-25: Some(k) makes prefill score every prompt position with its top-k alternatives (/v1/completions with echo and logprobs). Set by the scheduler before prefill. Such a request bypasses the prefix cache, so every position has a hidden row.

prompt_logprobs: Vec<PromptTokenLogprob>

2026-09-25: Filled across prefill chunks: one entry per prompt position i scoring tokens[i + 1]. The last prompt position, whose target is the first generated token, has no entry.

adapter_slot: i32

2026-09-25: The LoRA pool slot this sequence’s requests select (not slot_idx, which is the KV/SSM pool slot); -1 selects the installed active adapter. Set by the scheduler at prefill; the prefill, decode and verify paths route LoRA by it (moe_lora_route, upload_seq_slot_uniform).

acquired_adapter_slot: i32

2026-09-25: The resolved LoRA pool slot this sequence holds a ref on, as returned by Model::acquire_adapter_slot; -1 when none. free_sequence releases exactly this index and resets it to -1, so a rotation of the active adapter between prefill and finish cannot release a different slot.

src_lang_id: u32

2026-09-25: NLLB per-request source-language token id; 0 uses the deployment default.

tgt_lang_id: u32

2026-09-25: NLLB per-request target-language token id; 0 uses the deployment default.

num_beams: u32

2026-09-25: NLLB beam count; 1 turns beam search off.

length_penalty: f32

2026-09-25: NLLB length penalty: a hypothesis scores sum_logprob / len^length_penalty.

early_stopping: bool

2026-09-25: NLLB beam search: true stops once num_beams hypotheses have finished; false also waits until no running beam can beat the worst finished one.

Implementations

impl SequenceState

pub fn host_only(slot_idx: usize) -> Self

2026-09-25: A host-only sequence state: no GPU resources, no SSM slot, no layer states, every counter zeroed. NLLB’s alloc_sequence and the test models build on it; TransformerModel::alloc_sequence writes its own literal. Other crates can construct a SequenceState only through this, because ssm_slot is crate-private.

pub fn ssm_slot_idx(&self) -> Option<usize>

2026-09-25: The SSM pool slot this sequence holds, or None without one. The scheduler sorts decode batches by it.

pub fn hss_window_start(&self) -> usize

2026-09-25: Logical block index of block_table[0]: disk_block_ids.len() minus block_table.len(), saturating. 0 without --high-speed-swap, where disk_block_ids is empty.

pub fn physical_block_for(&self, abs_block_idx: usize) -> Option<u32>

2026-09-25: Physical HBM block of a logical block index, or None when that block has left the HBM window (or is past the end of block_table).