Expand description
2026-09-26: Warm and cold TTFT regression gates: one state machine, two modes.
- Warm: every sample at a prompt length sends the same prompt, and each measured request follows a priming request with that prompt.
- Cold: every sample gets a unique prefix tag, so no two requests share a prefix.
2026-09-27: The synthetic gates send prompt_lengths of filler; the
high-ISL gates send one committed long prompt and check the server’s
prompt-token count on every sample (ttft/long_prompt.rs).
Owner: bench, ttft. Invariants:
- The verdict compares the run’s median and p90 with the stored baseline of
the same box and model only; any other baseline gives an Info verdict
(
ttft_verdict.rs). - A failing run is never stored as the baseline (
should_store).
Structs§
Enums§
Constants§
- COLD_
DESCRIPTOR - COLD_
METADATA - HIGH_
ISL_ COLD_ DESCRIPTOR - 2026-09-27:
expected_secsbelow are one-shot runs (owner, 2026-09-27) at the 2026-09-27 measurements: one 32k-token prefill per cold run (MoE 14 s, dense 45 s), plus a priming prefill before the warm one. - HIGH_
ISL_ COLD_ MOE_ DESCRIPTOR - HIGH_
ISL_ WARM_ DESCRIPTOR - HIGH_
ISL_ WARM_ MOE_ DESCRIPTOR - WARM_
DESCRIPTOR - WARM_
METADATA