Module memory 

2026-10-02: The circuit memory model (owner directive 2026-10-02): for each node, the memory it holds (weights and their load-time copies, the activations it materializes at the peak run, its kernel family’s workspace, the state it touches), for each declared state and cache its bytes as a function of the runtime parameters, and their sum against a device’s budget.

• weights / copies: stored weights at the declared formats and the derived copies.
• caches: the cache kinds of crate::state::StateKind, sized from CacheInputs.
• budget: the class’s driver terms (HARDWARE.toml [memory]), the util budget and the inverse queries.
• render: the tables and the JSON of met circuit memory.

Owner: metrale-circuit (memory). Invariants:

• Pure (SBIO): every text, plan and count arrives as an argument. Counts the engine decides (slots, verify slots, snapshot slots, KV blocks, the legacy arena) are inputs, so the engine’s own functions decide them and this module only multiplies.
• Absent inputs size nothing; no count or format is defaulted here.
• Activations are the buffer planner’s arena for each run given (ActivationRun); the peak run sets the term. A legacy arena, when given, is what the budget charges instead (it is what the default forward allocates), and the workspaces it already holds (arena = true) are not charged twice. A workspace is sized at the widest run.
• A state is attributed to the node that owns it (updates or writes it); caches, which no node touches, appear only in the state table.

Re-exports

pub use budget::BudgetError;

pub use budget::DriverTerms;

pub use caches::CacheInputs;

pub use caches::CacheTerm;

pub use caches::LookupInputs;

pub use copies::CopyRule;

pub use copies::parse_copies;

pub use weights::NodeWeights;

Modules

budget

2026-10-02: The memory budget of a device class and the queries over it: the driver’s share (kernels/<class>/HARDWARE.toml [memory], calibrated per class), the util budget, and the inverse questions “the most sequences at this length” and “the longest length at this many sequences”, answered by bisection over the one evaluator the forward report uses.

caches

2026-10-02: The caches a circuit declares (StateKind::is_cache), sized: how many units each kind holds is a plan input (CacheInputs) the engine decides; the bytes of one unit are the declaration’s elements times its element size. A cache whose input is absent holds nothing; nothing here guesses a count.

copies

2026-10-02: Load-time derived weight copies (kernels/circuits/COPIES.toml): every device allocation the loader makes for a weight beyond the checkpoint tensors it stores (requantized serving copies, transposed prefill twins, repacks, widened scales, and the leaked intermediates nothing reads). A rule matches nodes by op, by the checkpoint’s declared weight format, by the served format and by serve settings, and states the byte law of its copy.

render

2026-10-02: met circuit memory’s output: a per-node table (one row per block-template node, summed over the layers that instantiate it, or every node with per_node), a per-state and per-cache table, the totals against the budget and the inverse queries; and the same report as JSON, every node and every state term listed.

weights

2026-10-02: Weight bytes per node: what the checkpoint stores for it (its declared formats, scales included) and every load-time copy the loader derives from it (super::copies).

Structs

ActivationRun

2026-10-02: One fused plan whose activations the model sizes, at its row count.

MemoryInputs

2026-10-02: Everything a memory model is evaluated from.

MemoryReport

2026-10-02: An evaluated memory model.

NodeMemory

2026-10-02: One node’s memory.

RunArena

2026-10-02: One activation run’s arena.

Totals

2026-10-02: The sums.

WorkspaceTerm

2026-10-02: One workspace of one family in a plan.

Enums

MemoryError

2026-10-02: Why a memory model could not be evaluated.

Functions

evaluate

2026-10-02: Evaluate the model.

kv_blocks_for

2026-10-02: KV blocks sequences sequences of tokens tokens each hold, as the scheduler’s admission reserves them (tokens / block_size + 1 per sequence: the block the next token writes into is reserved with the prompt), plus the pool’s one dummy block. None for a zero block size or on overflow.