Expand description
2026-09-25: Draft-count limits for speculative dispatch: the verify-slot capacity of every active sequence, and the serial path’s verify arm.
Owner: speculative. Invariants:
clamp_drafts_to_slot_capacityreturns no more thandraftsand no more than any capacity it is given.serial_verify_plankeeps at moststep_draftsof an MTP sequence’s pending drafts.
The verify pools size each slot for the deepest draft count the static
ladder can give a sequence in it (ssm_reserve::verify_slot_drafts).
With the default ladder 4:3,8:3,16:1,32:1 and --num-drafts 3, slots
0..8 hold 3 drafts and slots from 8 on hold 1. A step can ask for more:
a sequence can sit on a high slot while n_active is small, and
adaptive_rung can raise n in 9..=16 to 2 drafts. A step’s draft count is
therefore clamped to the smallest capacity among its active sequences,
read from the model’s pools (Model::mtp_slot_draft_capacity).
METRALE_MTP_POOL_FULL_WIDTH gives every slot num_drafts, which makes
the clamp a no-op. With the tiered default the adaptive lift never
survives the clamp: nine or more active sequences hold nine distinct
slots, one of them at index 8 or above, whose capacity is 1.
Enums§
- Serial
Arm - 2026-09-25: The verify arm the serial (per-sequence) path runs for one sequence.
Functions§
- clamp_
drafts_ to_ slot_ capacity - 2026-09-25: Clamps a step’s draft count to the smallest verify-slot
capacity among the active sequences.
usize::MAX(theModeldefault, for a model without SSM verify pools) does not clamp; an empty iterator leavesdraftsunchanged. - serial_
verify_ plan - 2026-09-25: How many of a sequence’s
pendingdrafts the serial path keeps, and which arm verifies them. An MTP sequence can carry more pending drafts than this step’s widthstep_drafts(the ladder count afterclamp_drafts_to_slot_capacity); the surplus is dropped so the verify never exceeds the slot. Withdflashevery pending draft is kept. Four or more kept drafts run the DFlash arm on a DFlash serve and the KN arm otherwise; fewer run K4, K3 or K2 by the kept count, each also bounded bystep_drafts.