Module gaps 

2026-09-26: Arrival-gap accumulator: the within-response jitter instrument.

An arrival is one socket read that carried at least one token delta (chat_stream_inner in http.rs); a gap is the time between two consecutive arrivals of one response. Gaps are taken per read, not per token: several deltas that arrive in one read are one arrival.

Count, mean and variance are kept online (Welford in push, Chan’s update in merge) with a running max. Percentiles need the samples, so gaps are retained as f32 up to RETAIN_CAP per response; past that the online statistics keep counting and GapStats::retained says how many gaps the percentiles cover. A cell pools its requests with GapSample::merge. stability and the coefficient of variation are derived from the primitives by GapStats; for both, lower is better.

Owner: bench (HTTP client). Invariants:

• A GapSample never retains more gaps than it has counted.
• GapSample::stats is None exactly when no gap was recorded.

Structs

GapSample

2026-09-26: Online accumulator of one response’s arrival gaps (or a pool of them).

GapStats

2026-09-26: One response’s (or one pool’s) arrival-gap distribution, in ms.

Constants

RETAIN_CAP

2026-09-26: Gaps retained per response for the percentiles. Equal to the concurrency driver’s largest osl, which is its max_tokens, so no concurrency response is truncated; a longer response keeps counting online and reports the truncation through GapStats::retained.

Functions

itl_ms

2026-09-26: Inter-token latency: decode_window_ms / (output_tokens − 1). The one definition both clocks use: the client passes its first-delta → stream-end window, and ChatOutcome::server_tpot_ms passes the endpoint’s usage.decode_time_ms.