Expand description
2026-10-02: Per-position confidence stop for MTP draft chains
(--draft-confidence-stop <tau>): the drafter keeps extending a chain only
while its last draft’s top-1 probability is at least tau. The serve
publishes tau once; the drafter (single and batched propose) and the
scheduler’s depth planner read it here and apply the one rule
chain_depth.
Owner: model-layers (speculative). Invariants:
- A published
tauis in(0, 1); a second publication of a different value is refused. chain_depthnever returns more than the drafts it is given, and returns at least 1 for a non-empty chain: the first draft is always verified, as without the stop.
Functions§
- chain_
continues - 2026-10-02: Whether a chain whose last draft scored
lpcontinues. - chain_
depth - 2026-10-02: How many leading drafts of a chain to verify, from each draft’s
top-1 log-probability
lps[j](in draft order): drafting continues past draftjonly whilelps[j] >= ln_tau, so the chain ends with the first draft below the threshold, which is still verified. An empty chain is 0. - draft_
stop_ logprob - 2026-10-02: The published threshold as a log-probability,
ln tau;Nonewhen the serve did not ask for the stop (and in unit tests and tools). - set_
draft_ confidence_ stop - 2026-10-02: Publish the serve’s stop threshold, before the model builds and the scheduler starts.