Module dflash_rung

Module dflash_rung 

Source
Expand description

2026-09-25: The DFlash gamma resolver: the per-step verify width K of a block-diffusion drafter, chosen by concurrency and, for a single stream, by the first-draft accept rate.

Owner: speculative. Invariants:

  • Unarmed (pinned, never configured, or a cap below 3), DflashRung::drafts_for returns num_drafts unchanged.
  • Armed, it returns K - 1 capped at num_drafts, and at least 1.
  • The single-stream state changes at most once per dwell single-stream verify steps.

K is Rungs::multi at C >= 2, and at C = 1 Rungs::wide or Rungs::narrow by the single-stream state (k_for). That state is decided on the popcount of a shift register holding one bit per C = 1 verify step (first draft accepted) over the last WINDOW steps: it goes wide at enter hits or more and narrow at leave hits or fewer (next_wide).

Measured on Qwen3.8-27B NVFP4 + DFlash-2 (block 8), one DGX Spark, 2026-09-03, tok/s:

Cprose (Volvo)code (MinHeap)
1γ5 27.0 / γ10 24.6γ10 66.8 / γ4 ~23
16γ4 180.7 / γ5 119.5γ4 279.5 / γ10 214.2

At C = 16, K = 4 won both workloads. K = 4 is the width of the write-on-accept verify kernel, which is opt-in (METRALE_GDN_WOA=1, gdn_flags::gdn_woa_enabled); without it Rungs::from_env puts the C >= 2 rung at the cap. At C = 1 prose preferred a narrow block and code the full block.

Measured at C = 1 on 2026-09-04 (MinHeap / Volvo): code accepted the first draft 0.97 of the time on the narrow rung and 0.875 on the wide rung, prose 0.56..0.76 on either. Code on the narrow rung lost about 18% (51 vs 63 tok/s), prose on the wide rung about 8% (24.8 vs 27.0). So LEAVE sits low: at 46, a 64-step window of code at 0.875 has 46 hits or fewer with probability about 6.5e-4 (binomial), and one of prose at 0.76 about 0.26.

DflashRung::configure leaves the resolver unarmed when the operator passes --dflash-gamma or sets METRALE_DFLASH_STATIC_GAMMA (presence); METRALE_DFLASH_GAMMA_RESOLVER (presence) arms it even under an explicit flag. It also reads the overrides, once, through Rungs::from_env: METRALE_DFLASH_RUNG_MULTI (K at C >= 2, default 4), METRALE_DFLASH_RUNG_NARROW (default 5) and METRALE_DFLASH_RUNG_WIDE (default the cap), each clamped to 2..=cap; METRALE_DFLASH_RUNG_ENTER (default 60), METRALE_DFLASH_RUNG_LEAVE (46) and METRALE_DFLASH_RUNG_DWELL (64).

Structs§

DflashRung
2026-09-25: The DFlash gamma resolver. The serve builds and configures one and the scheduler context holds it (SchedCtx::dflash_rung).
Rungs
2026-09-25: The resolver’s widths and thresholds. defaults and from_env clamp each width to 2..=max(cap, 2).

Constants§

WINDOW
2026-09-25: Window length in verify steps: every bit of the u64 shift register.

Functions§

k_for
2026-09-25: Verify K for n_active sequences, given the rungs and the single-stream state.
next_wide
2026-09-25: The next single-stream state from the number of first-draft hits in the last WINDOW steps.
shift_in
2026-09-25: Shifts one step into the register and returns the new register.