Module spec_capacity

Module spec_capacity 

Source
Expand description

2026-09-25: Draft-count limits for speculative dispatch: the verify-slot capacity of every active sequence, and the serial path’s verify arm.

Owner: speculative. Invariants:

  • clamp_drafts_to_slot_capacity returns no more than drafts and no more than any capacity it is given.
  • serial_verify_plan keeps at most step_drafts of an MTP sequence’s pending drafts.

The verify pools size each slot for the deepest draft count the static ladder can give a sequence in it (ssm_reserve::verify_slot_drafts). With the default ladder 4:3,8:3,16:1,32:1 and --num-drafts 3, slots 0..8 hold 3 drafts and slots from 8 on hold 1. A step can ask for more: a sequence can sit on a high slot while n_active is small, and adaptive_rung can raise n in 9..=16 to 2 drafts. A step’s draft count is therefore clamped to the smallest capacity among its active sequences, read from the model’s pools (Model::mtp_slot_draft_capacity). METRALE_MTP_POOL_FULL_WIDTH gives every slot num_drafts, which makes the clamp a no-op. With the tiered default the adaptive lift never survives the clamp: nine or more active sequences hold nine distinct slots, one of them at index 8 or above, whose capacity is 1.

Enums§

SerialArm
2026-09-25: The verify arm the serial (per-sequence) path runs for one sequence.

Functions§

clamp_drafts_to_slot_capacity
2026-09-25: Clamps a step’s draft count to the smallest verify-slot capacity among the active sequences. usize::MAX (the Model default, for a model without SSM verify pools) does not clamp; an empty iterator leaves drafts unchanged.
serial_verify_plan
2026-09-25: How many of a sequence’s pending drafts the serial path keeps, and which arm verifies them. An MTP sequence can carry more pending drafts than this step’s width step_drafts (the ladder count after clamp_drafts_to_slot_capacity); the surplus is dropped so the verify never exceeds the slot. With dflash every pending draft is kept. Four or more kept drafts run the DFlash arm on a DFlash serve and the KN arm otherwise; fewer run K4, K3 or K2 by the kept count, each also bounded by step_drafts.