Module ladder 

2026-09-25: Per-step MTP draft count as a function of the number of active sequences (the K-vs-batch ladder), and the multi-sequence MTP width cap.

Owner: model-layers (speculative). Invariants:

• mtp_ladder_drafts returns 0 when num_drafts is 0, and a value in 1..=num_drafts otherwise.
• The ladder, its kill switch and the cap are read from the environment once per process.

The default ladder is 4:3,8:3,16:1,32:1: three drafts up to 8 active sequences, one draft above that. A width above the last step takes the last step’s count. metrale_speculative::adaptive_rung uses this count as its floor and may raise widths 9..=16 to two drafts; it keeps the static count when METRALE_MTP_STATIC_RUNG or METRALE_MTP_K_LADDER is present.

Overrides:

• METRALE_MTP_K_LADDER, e.g. "4:3,8:2,16:1": comma-separated n_max:drafts steps, sorted by n_max. A value that does not parse as a whole gives the default ladder. Draft counts are clamped to 1..=num_drafts.
• METRALE_NO_MTP_K_LADDER, present with any value (0 included): every width gets num_drafts, and the default of mtp_max_seqs drops from 32 to 4.

Enums

MtpMaxSeqsSource

2026-09-29: Where the serve’s dispatch cap came from.

Constants

DEFAULT_MTP_MAX_SEQS

2026-09-29: Engine default of the multi-sequence MTP dispatch cap, when neither --mtp-max-seqs nor MODEL.toml [behavior] mtp_max_seqs gives one. 4 when METRALE_NO_MTP_K_LADDER is present.

MAX_MTP_MAX_SEQS

2026-09-29: The widest cap a serve accepts: one batched verify’s sequence width (layer::VERIFY_WY_TABLE_SEQS), which is also the serve’s widest batch.

Functions

ladder_drafts_from_steps

2026-10-03: The draft count of steps at n_active sequences under the num_drafts ceiling: the first step whose width covers n_active, else the last.

mtp_ladder_disabled

2026-09-25: Whether METRALE_NO_MTP_K_LADDER is present, with any value. Read once per process.

mtp_ladder_drafts

2026-09-25: The per-step draft count for n_active concurrent sequences.

mtp_ladder_pinned

2026-10-03: Whether the serve runs an explicit ladder (--mtp-k-ladder, or METRALE_MTP_K_LADDER present with any value): the adaptive rung then keeps the static counts, so the operator gets exactly the rungs spelled out.

mtp_max_seqs

2026-09-29: The multi-sequence MTP dispatch cap the serve published (set_mtp_max_seqs); before publication (unit tests, tools) the engine default.

parse_mtp_k_ladder

2026-10-03: Parses a --mtp-k-ladder value (n_max:drafts,...); an error names what does not parse, where the environment variable falls back to the default.

resolve_mtp_max_seqs

2026-09-29: Resolve the multi-sequence MTP dispatch cap: --mtp-max-seqs when given, else MODEL.toml [behavior] mtp_max_seqs when above 0, else the engine default (DEFAULT_MTP_MAX_SEQS, 4 under METRALE_NO_MTP_K_LADDER). A cap outside 1..=MAX_MTP_MAX_SEQS is refused, from either source.

set_mtp_k_ladder

2026-10-03: Publish --mtp-k-ladder once, before the model sizes its verify pools (ssm_reserve::verify_slot_drafts reads the ladder) and the scheduler starts. It wins over METRALE_MTP_K_LADDER. A second publication of a different ladder is refused.

set_mtp_max_seqs

2026-09-29: Publish the serve’s dispatch cap once, before the model sizes its verify pools and the scheduler reads its levers. A second publication of a different value is refused.