mtp_state_slots

Function mtp_state_slots 

Source
pub fn mtp_state_slots(max_batch_size: usize) -> usize
Expand description

2026-09-25: Number of SSM-pool slots the verify pools (per-token intermediates and the pre-verify checkpoint) cover.

Three callers use it: the server’s preflight_reserve sizes the pre-load reserve with it, SsmStatePool::new allocates that many verify slots plus a dummy, and for a model with SSM layers the scheduler takes the plain-decode branch while any active slot is at or above it (spec_slot_cap, lane_decode.rs). Retirement compacts survivors onto low slots, except under EP protocol v2 (retire_finished_sequences).

It is max_batch_size when that is at most 32 (MTP_STATE_SLOTS_FLOOR, the floor) or when mtp_pool_full_width holds; otherwise max(mtp_max_seqs(), 32), capped at max_batch_size. The scheduler never speculates with more than mtp_max_seqs() active sequences.