Module dataset

Module dataset 

Source
Expand description

2026-09-26: The MLPerf Agentic Inference trajectory file: flat JSONL, one row per message, each conversation’s rows contiguous and numbered 1..=N in file order.

Teacher forcing: the prompt for a client turn is the recorded history (system prompt and recorded user, tool and assistant rows, including recorded reasoning_content and tool_calls) plus the current client message, prebuilt at load time. The model’s own output is never fed into later turns, so per-turn prompts depend only on the file.

Owner: bench, mlperf_agentic. Invariants: load returns Ok only for a file whose every conversation passes the id, contiguity, turn-number and role checks, and a non-empty draw.

Structs§

ClientTurn
2026-09-26: One client turn: what is sent, and the recorded assistant row that scores it.
Conversation
DrawSpec
2026-09-26: How many whole trajectories to take per domain, the first K in file order, with no RNG. 0 means all of that domain. Conversations are drawn whole.

Functions§

draw_fingerprint
2026-09-26: The draw fingerprint: SHA256 over the ordered conversations’ ids, tools, and each client turn’s number, messages and ground truth. Two runs with equal fingerprints replayed the same prompts, tools and ground truth in the same order; a differing fingerprint means the numbers are not comparable, whatever the sample counts say.
load
2026-09-26: Load the trajectory file and apply the draw.