Expand description
2026-09-26: The serve matrix’s bars and tally, as pure functions of the round outcomes.
Owner: bench, serve matrix. Invariants:
- No I/O.
- An identity, codegen or tool-call signal that is
NotRunfails its bar, and a planned round with no result fails asno-result.
Structs§
- Round
Result - 2026-09-26: One planned round’s result.
- Signals
- 2026-09-26: What one booted round measured.
- Tally
- 2026-09-26: The whole matrix, scored.
Enums§
- Outcome
- 2026-09-26: What became of a planned round.
- Signal
- 2026-09-26: One probe’s answer.
NotApplicablepasses only the tool-call bar, where it means the server refused the tools request with a 4xx; on the identity and codegen bars it fails.
Constants§
- COHERENCE_
MIN_ PASS - 2026-09-26: Coherence probes that must pass.
barscaps it at the number run, so with fewer probes every one must pass. - TPS_
TOLERANCE - 2026-09-26: Fraction below a stored baseline tok/s that counts as a regression.
Functions§
- tally
- 2026-09-26: Score every planned round against its result. Walks the plan, not
the results, so a planned round with no result fails as
no-result. - verdict_
text - 2026-09-26: The verdict sentence: coverage, skips (up to three named), the filter count, and each failing round’s bars.