Module speculative 

2026-09-25: The draft-proposer interface for speculative decoding (DraftProposer, ProposerState), and the MTP knobs, ladder, tree shape and verify keys in its submodules.

Owner: model-layers (speculative). Invariants: none beyond the types.

Re-exports

pub use ladder::DEFAULT_MTP_MAX_SEQS;

pub use ladder::MAX_MTP_MAX_SEQS;

pub use ladder::MtpMaxSeqsSource;

pub use ladder::ladder_drafts_from_steps;

pub use ladder::mtp_ladder_disabled;

pub use ladder::mtp_ladder_drafts;

pub use ladder::mtp_ladder_pinned;

pub use ladder::mtp_max_seqs;

pub use ladder::parse_mtp_k_ladder;

pub use ladder::resolve_mtp_max_seqs;

pub use ladder::set_mtp_k_ladder;

pub use ladder::set_mtp_max_seqs;

Modules

draft_stop

2026-10-02: Per-position confidence stop for MTP draft chains (--draft-confidence-stop <tau>): the drafter keeps extending a chain only while its last draft’s top-1 probability is at least tau. The serve publishes tau once; the drafter (single and batched propose) and the scheduler’s depth planner read it here and apply the one rule chain_depth.

ladder

2026-09-25: Per-step MTP draft count as a function of the number of active sequences (the K-vs-batch ladder), and the multi-sequence MTP width cap.

tree_shape

2026-09-25: Spine-plus-hedge draft trees for tree speculative decoding: the tree shape, its verify-row layout and the accepted path.

verify_key

2026-09-25: Batched-verify ordering and graph key: the dispatch order and depth assignment of one verify batch, and the CUDA-graph cache key built from them.

Constants

EP_CMD_MTP_PROPOSE

2026-09-25: EP worker command: run one MTP propose alongside rank 0. The code is followed by last_token, position and num_drafts, each a u32.

Traits

DraftProposer

2026-09-25: A draft-token proposer. The model calls propose for draft tokens, verifies them with the target, then calls after_verify with the accepted count so the proposer can trim its state.

ProposerState

2026-09-25: Per-sequence state of a DraftProposer, reached through as_any downcasts like LayerState.

Functions

hidden_fingerprint

2026-09-25: FNV-1a over the h BF16 values at p; 0 when the copy to the host fails.

mtp_accept_debug

2026-09-25: METRALE_MTP_ACCEPT_DEBUG, on when set to any value. It turns on MTP acceptance logs, among them the periodic per-batch-width lines of the scheduler’s mtp_accept_debug module, verify-graph outcome counts and a drafter coverage line. Read once per process.

mtp_catchup_enabled

2026-09-25: METRALE_MTP_CATCHUP=1: the model keeps a ring of target hiddens, and before a propose it feeds the drafter the pair keys missing since its newest row. Off otherwise, and off in mtp_multi_seq_mode.

mtp_ep_propose_enabled

2026-09-25: Run the sharded drafter on every rank with the communicator. On unless METRALE_NO_MTP_EP_PROPOSE=1; read once per process. It is the GLM-5.3 MTP head’s DraftProposer::needs_comm, and a proposer that needs the communicator also has the worker run each propose (EP_CMD_MTP_PROPOSE).

mtp_multi_seq_mode

2026-09-25: True when the MTP dispatch cap is above one sequence. The catch-up ring, the refeed labels and the carry slot each hold one sequence’s data, so mtp_catchup_enabled, mtp_refeed_accepted_enabled and mtp_carry::carry_armed are false in this mode.

mtp_refeed_accepted_enabled

2026-09-25: METRALE_MTP_REFEED_ACCEPTED=1: after a verify, ring the target’s hiddens for the accepted positions, so the catch-up feed can rewrite accepted drafter rows with the target’s hidden instead of the drafter’s own. It writes into the catch-up ring, which the model allocates only when mtp_catchup_enabled. Off otherwise, and off in mtp_multi_seq_mode. Measured 2026-07-21 on dgx2 at 2 drafts, about 10k verify steps per arm: p2_uncond 0.4182 off, 0.4452 on.

mtp_refeed_debug

2026-09-25: METRALE_MTP_REFEED_DEBUG=1: log a hidden_fingerprint of each hidden written to or fed from the catch-up ring. Each one copies a row to the host.

mtp_refeed_shift

2026-09-25: METRALE_MTP_REFEED_SHIFT: an offset added to every re-fed ring label, clamped to [-4, 4]; 0 (unset or unparseable) is the derived mapping. It exists to test that mapping: a run at +1 or -1 is compared with 0 by acceptance.

shadow_topk

2026-09-25: METRALE_MTP_SHADOW_TOPK=k: log the drafter’s top-k candidates at each draft position, and the verify steps’ target tokens, without changing token selection. 0 (unset or unparseable) is off; k is capped at 8. ModelLevers::shadow_topk and SchedLevers::shadow_topk both read it here, once per run.