Module adaptive_rung 

2026-09-25: Adaptive MTP draft count for batch widths 9..=16: one or two drafts, chosen from the observed accept statistics, with the static ladder (metrale_model_layers::speculative::mtp_ladder_drafts) as the floor.

Owner: speculative. Invariants:

• drafts_for never returns fewer drafts than the static ladder, and returns exactly the ladder’s count outside 9..=16, when RungParams::disabled, when METRALE_NO_MTP_K_LADDER is set, or when num_drafts < 2.
• observe changes no state for widths outside 9..=16 or when disabled.

With p1 the first-draft accept rate and p2_cond the second-draft accept rate given an accepted first draft, two drafts yield token_ratio(p1, p2_cond) = 1 + p1*p2_cond/(1 + p1) times the tokens per verify step of one draft. The controller moves to two drafts when the smoothed ratio reaches enter and back to one when it falls below leave (next_state). The estimates are EWMAs seeded with their first sample. With the default tiered verify pools the scheduler clamps the lift back to one draft (spec_capacity::clamp_drafts_to_slot_capacity; see that module).

At one draft nothing proposes a second token, so p2_cond cannot be observed there. The controller then runs two drafts for a flush (a probe) only on evidence or after a long backstop; see drafts_for. Measured 2026-08-01 on dgx1 at C=16 from flush timestamps: a steady flush took 1.16 s and the flush that entered a probe 2.90 s, so a probe cost about 1.74 s.

Structs

AdaptiveRung

2026-09-25: The controller. Each scheduler context owns one (SchedCtx::rung), so accept history is not shared between contexts.

RungParams

2026-09-25: The controller’s thresholds, fixed for the life of one AdaptiveRung.

Functions

next_state

2026-09-25: The next state from the current one and the smoothed token ratio; true means two drafts.

p2_cond_from

2026-09-25: The second-draft conditional accept implied by a flush at two drafts, where mean_na = p1 + p1*p2_cond: (mean_na - p1) / p1, clamped to [0, 1]. Deeper drafts are not modelled. None when p1 <= 0.

token_ratio

2026-09-25: Expected tokens per verify step at two drafts relative to one: (1 + p1 + p1*p2_cond) / (1 + p1). Returns 1.0 when p1 <= 0 or either input is not finite.