Module rank_agree

Module rank_agree 

Source
Expand description

2026-09-25: Startup check that every rank holds the same values for the env-read scalars that shape the collective schedule.

Such a scalar is read independently on each rank. For example, metrale_model_arch::glm5next_layer::prefill_rows (METRALE_GLM_PREFILL_ROWS) sets the row count of each prefill sub-chunk, and each sub-chunk issues its own reduce_partial, so a skew between ranks is a hang or a reduce over the wrong extent rather than a perf difference. Rank 0 broadcasts its values, every rank compares them with its own, and a mismatch fails model construction with an error naming each disagreeing entry. The caller, TransformerModel::new, chooses the list.

Owner: model-engine. Invariants: none beyond the types.