Module mtp_gate 

2026-09-25: The MTP runtime gate: it chooses, step by step, between the MTP verify step and plain decode by the delivered throughput (emitted tokens per second of step wall time) each mode measured over windows of WINDOW_STEPS steps.

Owner: speculative. Invariants:

• The mode changes only in arbitrate, and only when both modes have an estimate, the other mode’s is not stale, and it beat the current mode by the margin in SWITCH_DWELL_WINDOWS consecutive arbitrations.
• A window never mixes power-of-two batch-width buckets: note_width discards the partial window before a step from a new bucket is added.

Policy:

• Run the current mode, and fold each closed window into that mode’s tok/s EWMA and deviation EWMA (ModeStats::update).
• In Serial mode, probe MTP for one window after reprobe_tokens recorded tokens; in Mtp mode, refresh the serial estimate after serial_refresh_tokens.
• A depth-regime change (factor REMEASURE_DEPTH_FACTOR, floor REMEASURE_DEPTH_FLOOR, maybe_remeasure) or a change of power-of-two batch-width bucket (note_width) marks both estimates stale and brings the next probe forward. A stale other-mode estimate cannot win a switch.
• A plain decode step over n sequences is charged n tokens, and a verify step the tokens all its sequences emitted, so both modes are measured as whole-batch throughput.

With --mtp-gate force (or METRALE_MTP_GATE_FORCE) the scheduler builds no gate and runs the MTP step whenever speculation is eligible.

Structs

MtpGate

2026-09-25: The gate. The scheduler builds one when MTP is on and the gate is not forced off, and records each gated plain-decode or MTP step in it; steps run under a pin are not recorded.

Enums

GateDecision

2026-09-25: A mode switch, handed to the scheduler once (take_fresh_decision).

GateStep

2026-09-25: The step the gate wants the scheduler to run next.

Functions

entry_pin_forces_verify

2026-09-25: Whether the spec-entry pin overrides a Serial gate decision for this step. min_post_think_emitted is the minimum over the active batch, so one entering sequence pins the whole (already spec-eligible) batch. pin_tokens is the run’s entry_pin_tokens_from_env value.

entry_pin_tokens_from_env

2026-09-25: The pin width from METRALE_SPEC_ENTRY_PIN. The scheduler stores it in SchedLevers::spec_entry_pin_tokens.

spec_dispatch_eligible

2026-09-25: Whether one sequence may take the speculative path this step. Never with suppress_tool_call or disable_mtp, and never inside <think> unless spec_think (METRALE_DFLASH_SPEC_THINK=1), for MTP and DFlash alike. Otherwise it needs resume_guard emitted tokens: counted after </think>, except inside <think> (reachable only with spec_think), where the whole output counts.