Expand description
2026-09-26: The agentic benchmark’s agent loop: tool-calling turns against the
served endpoint, with the tools run inside a per-iteration sandbox directory.
Its tools, prompt and caps are modelled on the opencode client that
bench/fp8_dgx2_drift/harness/run_tier.sh drives.
Owner: bench, agentic. Invariants:
- The loop runs at most
max_turnsturns. - Every tool result sent back to the model has passed through
truncate. run_taskreaps processes whose working directory is in the sandbox after the loop returns, whether it returnedOkorErr.
Re-exports§
pub use shell::truncate;pub use tools::glob_match;pub use tools::tool_schema;
Modules§
- norm
- 2026-09-26: Normalising
bashtool output so that identical work reads identically: progress lines are dropped, and timestamps, dates, durations, pids andpsbookkeeping columns become placeholders. Every tool output is fed back to the model, so a build that reportsin 0.00son one run andin 1.23son the next gives two greedy runs different contexts. - shell
- 2026-09-26: Running the shell command the model wrote, and bounding its time, memory and output.
- tools
- 2026-09-26: The six tools the agent loop offers (
bash,read,write,edit,glob,grep) and their implementations. - trace
- 2026-09-26: Per-iteration trajectory file: the prompt, then each turn’s
reasoning, text, tool calls and finish reason, and each tool result, as
plain text so
diffof two runs shows the first turn at which they split. The outcome columns (webserver_ok, steps, turns) can agree while the trajectories differ.
Structs§
- Agent
Config - Transcript
- 2026-09-26: What one agent run did, for scoring and diagnostics.
Functions§
- resolve
- 2026-09-26: Resolve
pathinsidesandbox. The check is lexical, because the target usually does not exist yet: a relative path, or an absolute one already undersandbox, is accepted unless it has a..component or resolves, through an existing symlink, outside the sandbox. - run_
task - 2026-09-26: Run one agentic task until the agent stops calling tools or the turn cap is reached.