Expand description
2026-09-26: Path-independence of tool calls: one target turn, whose correct answer is to call nothing, reached by three predecessor paths that share the same long prefix. The replay script offers no tools, and its comparison passes reworded prose, so it cannot see a tool call that moves with history.
Owner: bench, SSM poisoning gate. Invariants:
- Only the target’s tool calls are compared, raw and in order; its prose is not.
A prefix → ack → TARGET
B prefix → ack → a turn that calls get_current_weather → TARGET
C prefix → ack → a turn that calls web_search → TARGETThe user turns of the paths differ only in the interposed turn. Path A is
the reference; the target’s tool_calls must equal A’s on B and C. The
target asks for something no offered tool answers, so a call there is wrong
whatever its wording.
Structs§
- Divergence
- 2026-09-26: One path disagreeing with the reference.
- Path
Result - 2026-09-26: One path’s outcome: the target turn’s transcript.
Enums§
Constants§
- CALLS_
SEARCH - 2026-09-26: The interposed turn of path C, which asks for a
web_searchcall. - CALLS_
WEATHER - 2026-09-26: The interposed turn of path B, which asks for a
get_current_weathercall. - TARGET
- 2026-09-26: The target turn. Its answer is in the document the conversation already carries, and neither offered tool bears on it, so the correct reply calls nothing.
Functions§
- divergences
- 2026-09-26: Compare every path against the reference (
Path::Direct). Returns oneDivergenceper path whose tool calls differ at all; there is no tolerance band as there is for replay prose. Empty when every path agrees, and also when no reference path is present. - reference_
called - 2026-09-26: Did the reference path itself call a tool on the target? A
different finding from
divergences: the driver fails the run on a divergence and only logs this one. - request_
body - 2026-09-26: The request body for a tool-offering turn: the replay probe’s
greedy, seed-0 body plus
toolsandtool_choice: "auto", so the model decides whether to call. - tools
- 2026-09-26: The tool schemas offered on every turn of this probe.