Expand description
2026-09-26: Quick Speed Bench: warmup plus N timed streaming runs of one
prompt, reporting the server’s decode rate, TTFT and end-to-end latency. A
measurement tool, not a gate: it stores no baseline, reads no threshold,
and gate::coverage::NOT_REQUIRED lists it.
Which tok/s to quote:
- Decode tok/s (server), the headline:
usage."response_token/s", which the server computes as(completion_tokens - 1) / decode_time(ir::Usage::decode_rate_tok_s). It excludes prefill, client buffering and the network. - E2E tok/s (client):
completion_tokens / wall, including prefill and transport.
TPOT is derived from the server’s decode rate (1000 / server tok/s), or
omitted when the server reports none; no client-clock TPOT is emitted,
because buffered reads and multi-token emission compress client-side
arrival spacing.
Every run, warmup included, sends the same prompt, so the timed TTFT is a
warm figure. ttft-warm-gate and ttft-cold-gate control the prefix.
Owner: bench, quick_speed. Invariants: each average counts only the runs that reported its value.