Module draw 

2026-09-26: The BFCL sample draw. normalized_single_turn_score depends on the category mix, so a score is comparable only with scores on the same draw.

Two deterministic rules, with no RNG, produce it:

1. Selection by category. Asking for [non_live, live, hallucination] expands to those categories’ subsets; live_relevance belongs to none of them, so it drops out. That exclusion is the difference between n = 1011 and the golden n = 995.
2. Per-subset head(n). n = total if total <= subset_floor else max(1, int(total * pct / 100)), taking the first n rows, concatenated in sorted subset order.

Owner: bench, BFCL benchmark. Invariants: shard_take over 0..count sums to take for every count of at least 1.

Structs

DrawSpec

2026-09-26: Sampling configuration.

Constants

CATEGORIES

2026-09-26: The three scored categories.

MINI_SAMPLES

2026-10-03: The sample count of DrawSpec::mini on reference_subset_totals.

SINGLE_TURN_SUBSETS

2026-09-26: Every single-turn subset, in the order provision.py writes them.

Functions

category_of

2026-09-26: Which category a subset belongs to. live_relevance maps to None: score.py scores it per sample but in no category, and a category selection leaves it out.

fingerprint

2026-10-03: The identity of the draw a run scored, for its record’s dataset_fingerprint: draw-sha256:<hex>;n=<N>;category_sample_pct=<cat>:<pct>,...;subset_floor=<f>. The hash is over the run’s sample ids in the order they were issued, one per line, so two runs share a fingerprint only when they scored the same samples in the same order; a BFCL score is not meaningful without it. A shard’s fingerprint covers the shard’s own slice.

plan

2026-09-26: Applies the draw to per-subset totals, returning (subset, take) sorted by subset name and without subsets that take nothing.

reference_subset_totals

2026-09-26: The BFCL v4 single-turn row counts per subset, the table the draw and shard tests and Bfcl::expected_samples use. provision.rs saves the counts provision.py wrote as dataset_summary.json; they are expected to equal these.

shard_owns

2026-09-26: Whether row row of a subset (0-based, within the drawn rows) is in shard index of count.

shard_plan

2026-09-26: The per-subset counts one shard runs, without empty subsets.

shard_take

2026-09-26: How many of a subset’s take drawn rows shard index of count gets; 0 for an index out of range or a count of 0. Rows are striped, row i going to shard i % count (shard_owns), not cut into contiguous blocks, so every shard gets a proportional slice of every subset, and membership depends on position alone.

total

2026-09-26: Total sample count for a plan.