Module verify_key 

2026-09-25: Batched-verify ordering and graph key: the dispatch order and depth assignment of one verify batch, and the CUDA-graph cache key built from them.

Owner: model-layers (speculative). Invariants:

• verify_batch_permutation returns a permutation of 0..n, and panics when slots and ks differ in length.
• With canonical = true, verify_batch_order places slots in non-decreasing order and depths in non-increasing order, and returns the multiset of depths it was given.
• With canonical = false, each member keeps its own depth, ordered deepest first, then by slot.

The scheduler orders each batch with these functions (mtp_dcut::plan, mtp_step) and the model keys its batched-verify graphs with verify_graph_key (verify_e2::verify_batched_graph_key), so the dispatch and the key see one order. Under the canonical assignment the key depends on the slot set and the depth multiset, not on which sequence D-Cut gave which depth, which reduces the number of distinct keys. Ascending slots also keep a depth run on consecutive slots when the pool slots are consecutive; the batched GDN path checks the actual pointers and declines otherwise (trait_decode_batched_conv_gdn_multi.rs).

canonical_assignment chooses the arm from the batch width. METRALE_NO_CANONICAL_VERIFY_KEY, present with any value (0 included), turns the canonical arm off at every width.

Constants

CANONICAL_KEY_MIN_WIDTH

2026-09-25: Batch width (sequences) at or above which canonical_assignment selects the canonical arm, unless METRALE_CANONICAL_KEY_MIN_WIDTH overrides it. Above the scheduler’s D-Cut width cap (SchedLevers::dcut_width_cap), mtp_dcut::plan leaves every depth equal, and then both arms give the same order (uniform_depths_are_identical_under_both_arms).

Functions

canonical_assignment

2026-09-25: Whether a batch of n sequences gets the canonical depth-to-slot assignment. mtp_dcut::plan asks with the batch width and mtp_step asks with the same width for each chunk it dispatches, so the order plan assigned and the permutation mtp_step applies use the same arm.

canonical_key_min_width

2026-09-25: The width threshold: METRALE_CANONICAL_KEY_MIN_WIDTH when it parses as a usize (0 selects the canonical arm at every width), else CANONICAL_KEY_MIN_WIDTH. Read once per process.

canonical_verify_key_enabled

2026-09-25: True unless METRALE_NO_CANONICAL_VERIFY_KEY is present. Read once per process.

verify_batch_order

2026-09-25: Orders one verify batch and assigns its depths; the entry point of mtp_dcut::plan, which truncates each sequence’s drafts to the depth assigned here.

verify_batch_permutation

2026-09-25: Dispatch order for one verify batch, as a permutation only.

verify_graph_key

2026-09-25: The batched-verify CUDA-graph cache key: the (ssm slot, row count) pairs in dispatch order, then one sentinel word, u32::MAX - wy_tables_null - 2 * write_on_accept, which differs for each of the four flag combinations. write_on_accept is in the key because it can select a different WY launch (the write-on-accept K=4 twin, trait_decode_batched_conv_gdn_multi.rs).