Module draft_stop

Module draft_stop 

Source
Expand description

2026-10-02: Per-position confidence stop for MTP draft chains (--draft-confidence-stop <tau>): the drafter keeps extending a chain only while its last draft’s top-1 probability is at least tau. The serve publishes tau once; the drafter (single and batched propose) and the scheduler’s depth planner read it here and apply the one rule chain_depth.

Owner: model-layers (speculative). Invariants:

  • A published tau is in (0, 1); a second publication of a different value is refused.
  • chain_depth never returns more than the drafts it is given, and returns at least 1 for a non-empty chain: the first draft is always verified, as without the stop.

Functions§

chain_continues
2026-10-02: Whether a chain whose last draft scored lp continues.
chain_depth
2026-10-02: How many leading drafts of a chain to verify, from each draft’s top-1 log-probability lps[j] (in draft order): drafting continues past draft j only while lps[j] >= ln_tau, so the chain ends with the first draft below the threshold, which is still verified. An empty chain is 0.
draft_stop_logprob
2026-10-02: The published threshold as a log-probability, ln tau; None when the serve did not ask for the stop (and in unit tests and tools).
set_draft_confidence_stop
2026-10-02: Publish the serve’s stop threshold, before the model builds and the scheduler starts.