The Circuit Compiler

The circuit compiler turns a checkpoint into the exact list of kernel launches that run it on a given hardware class. It is the crate metrale-circuit (crates/circuit) and the met circuit commands built on it. The crate is pure: it does no GPU work, file I/O, environment reads, clock reads or random draws. Callers read the TOML files and pass their text in, and every plan is deterministic and carries a digest.

This chapter names each stage and the vocabulary the rest of the book and the code use for it. The Latent Kernel Blueprint covers the kernel side in depth. The Latent Architecture Blueprint covers the model side. The LKB in Mathematics states both precisely.

The stages

config.json ──config map──► circuit instance ──lowering──► kernel groups ──realization──► plan
 (checkpoint)   (<arch>.config.toml)  (a Σ-diagram)   (fuser + FUSIONS.toml)  (LKB points)  (class H)
                                                                                         │
                                                         circuit executor (--forward circuit)

Tensor types

An edge carries a tensor type: rows × a dim expression × a format (crates/circuit/src/format.rs). A format carries its scale granularity (per tensor, per token, per channel, per group or per block). Memory layout is not yet part of the type. Layout choices are made inside kernels and in the buffer planner, which assigns arena offsets and is a different thing from tensor layout.

Lowering rules and their numerics tags

A rule in FUSIONS.toml is a lowering clause. Under the modes, row counts and serving settings it names, it maps a pattern of ops to one or more kernels. The fuser applies rules greedily by priority, and every node ends up in exactly one group. Every rule carries a numerics tag that states how its kernel relates to running the same ops one by one:

The tag is part of the plan digest. met circuit diff checks byte parity between the hand-written forward and the circuit forward: first with reference rules only, then with every rule.

When a relation may become the default

A bit_identical rule produces the same bytes as the chain it replaces, so it cannot change accuracy. It may therefore become the default lowering (its when setting turned on in the serving policies) on two pieces of evidence, with no accuracy gate:

1. Byte identity. Its microtest passes, and met circuit diff shows identical logits with the rule on and off.
2. No regression in speed and no regression in energy. Measure with a same-box A/B: the same binary, the same box, the rule on vs off, across the concurrency ladder the gates use, with a control leg. Speed and energy are measured separately (J = ∫ P dt; one does not imply the other).

A relation that wins on one and loses on the other stays opt-in, as a flagged choice that records both deltas. The same rule appears in the FUSIONS.toml schema notes and in ADR-0018.

Hardware classes and realization

A hardware class (kernels/<class>/HARDWARE.toml) names its architecture and the class it inherits from. It also holds its serving defaults and its tensor-core policy. Its common/FUSIONS.toml and common/KERNEL_FAMILIES.toml may inherit the parent's and override or remove entries. The plan on a class uses the rules and families resolved along that chain, restricted to kernels the class can run (crates/circuit/src/hardware/avail.rs).

The tensor-core policy is a constraint on the realization. It lists which ops must run on tensor cores at which rows and formats, and every exemption is explicit. A plan that violates it is refused.

Realization is exact for the plan's structure: which kernels, composed how. It is exact for the plan's numbers only where a class keeps the same reduction order as the reference class. See parity tiers.

Reading the reports in this vocabulary

The LKB chapter lists what the reports do not compute yet, notably the LKB residual.

Recipes: a parameter point plus a serving policy

This is the target design. It is adopted, and existing recipes migrate to it in a later change. A launch recipe is made of exactly three parts:

1. The checkpoint's parameter point. The architecture circuit and its dims, switches and precision are derived from the checkpoint by the config map. A recipe names the checkpoint and never restates its architecture.
2. A serving policy. These are the choices that may change what is computed or how the serve behaves, each stated explicitly:
   • the precision tier (declared, or a disclosed lower tier);
   • KV-cache format;
   • speculative decoding (draft, depth);
   • batch and KV limits;
   • the lowering settings that choose between byte-different kernels (see below);
   • opted-in levers.
3. Resources. Devices, nodes, memory utilization and ports.

Choices that preserve semantics never appear in a recipe. Fusions, projection concatenations, scheduling and stream assignment produce the same bytes either way, so they are the compiler's to make, chosen by cost.

Which lowering settings belong in a recipe is derived from the rules, not decided by hand. A rule's when key selects among rules. A key belongs to the serving policy when the rules it selects among can produce different bytes: a reference rule against another reference rule, or any differs rule. A key that only switches bit_identical rules on or off belongs to the compiler, and it becomes a default under the rule above.

The recipe checker requires every recipe-settable key to be explicit, so no serve default is inherited silently. Under this design, the lowering part of its key list is computed from the rules in this way rather than maintained by hand. The design note recipe shape gives the full mapping and the migration order.

Vocabulary in the code