Expand description
2026-09-26: Serve Matrix: boot every checkpoint the box can serve, run the coherence, codegen, tool-call, long-context and throughput probes on it, and score one row per round.
Owner: bench, serve matrix. Invariants:
- The rounds come from the host’s roster (
ServeHost::roster); this crate lists no model. - Scoring walks the plan, not the results, so a planned round with no result
fails (
score::tally).
Modules§
- host
- 2026-09-26: The seam the serve matrix boots checkpoints through: the serving
process installs a
ServeHost, because this crate starts no server. - plan
- 2026-09-26: Turning the box’s roster into the rounds this run intends to cover. A round is planned (the run boots it and it must produce a result), skipped (the box cannot serve it; counted and reported) or excluded (outside the operator’s filter).
- probes
- 2026-09-26: The serve matrix’s per-round probes: each turns its requests into
a
Signal(or a tok/s figure); the bars that read them are inscore.rs. - report
- 2026-09-26: Rendering the matrix: one table row per roster entry, one tile row of counts, the verdict, and the per-round log line.
- schema
- 2026-09-26: The serve matrix’s parameter schema.
- score
- 2026-09-26: The serve matrix’s bars and tally, as pure functions of the round outcomes.