Expand description
2026-09-26: Scoring for the contamination benchmark: classify every prompt at
every rung and in the post-check against its own solo reference. The module
reads no handle, network or clock; score_tests.rs tests it with
in-memory outcomes.
Why more than “one solo reference, then diff each rung against it”:
- A single reference cannot tell a concurrency defect from a prompt that
is not reproducible alone. The reference runs twice; a prompt whose two
solo runs disagree is
AloneUnstable, is excluded from the per-rung comparison, and still fails the verdict under its own name. - State that survives the concurrent episode shows only in later solo
work, so a solo post-check leg after the rungs classifies a divergence
there as
Persistent. - The driver primes every prompt once before any measured leg, so the
measured legs start cache-warm unless a prime request failed (the driver
then logs a warning).
cached_prompt_tokensis not part of the comparison.
Owner: bench (contamination). Invariants:
- Each cell inserted into
Score::cellsincrements exactly one of the class counters. verdictpasses only when at least one cell was compared and every compared cell isIdentical.
Structs§
Enums§
- Class
- 2026-09-26: What happened to one prompt in one leg: a rung, the
post-check, or (as
"ref") its reference.