Module spec_cost

Module spec_cost 

Source
Expand description

2026-10-04: Speculative step cost calibration: on the serve it is pointed at, the wall time and GPU-rail energy of ONE speculative scheduler step at each batch width n, for the draft depth k the serve runs, split into its draft (propose) and verify shares. A measurement tool for the speculative cost model, not a gate: gate::coverage::NOT_REQUIRED lists it.

Per width, ascending (wave.rs): n concurrent streaming essay requests, each with its own prompt identity; once every stream has delivered a token and settle_s has passed, two /metrics snapshots window_s apart and the rail energy between them; then the streams are closed and the serve drained to idle. The arithmetic, and its proportional energy split, are in cell.rs.

The pins (the benchmark’s definition, not parameters):

  • Prompt: wave::PROMPT_TOKENS of stats::make_prompt filler, then the concurrency sweep’s essay ask (concurrency::ESSAY_TASK), which keeps a model writing.
  • Request: temperature 0, seed 0, presence and frequency penalty 0, and reasoning_effort: "none" (thinking off), as in decode_floor: speculative dispatch is off inside <think> unless METRALE_DFLASH_SPEC_THINK=1.
  • A speculative serve (k > 0) must run with --telemetry basic or kernel, which exports the scheduler phase histogram; without it the run fails.

Owner: bench, spec-cost. Invariants: the run verdict is never a pass: info when at least one width was measured, INCONCLUSIVE otherwise.

Modules§

acceptance
2026-10-04: The run’s acceptance counts, for the drafter’s confidence→acceptance calibration: between the probe’s /metrics page and the one after the last width, the change in
table_input
2026-10-04: Reads a spec-cost run’s record back into per-width step costs, the input of met benchmark spec-cost-table. The writer is cell::record; the key names are shared with it. Pure: no I/O.

Structs§

SpecCost

Constants§

DESCRIPTOR
METADATA