Module factory

Expand description

2026-09-25: Model factory: loader_for_config picks the ModelWeightLoader for a config’s model_type, and build_model builds the model from the config and the weights.

Owner: metrale-model-engine. Invariants: none beyond the types.

Structs§

BuiltModel: 2026-10-01: A built model and the slot count it was built with (the request, or what auto resolved to), which the scheduler and the serve’s disclosure take from here.
DflashBuildArgs: 2026-09-25: DFlash drafter inputs for build_model: the drafter’s own WeightStore, its parsed config.json, and the CLI overrides for γ and the sliding-window size.
LoraBuildArgs: 2026-09-25: LoRA inputs for build_model (--lora-adapter NAME=PATH): the adapters and the pool-shape CLI values.
SlotPlan: 2026-10-01: The slot request and the pre-load reserve as a function of the slot count (the preflight’s plan, inference_reserve(slots)), which KV sizing evaluates at the count it builds.

AUTO_MAX_SLOTS: 2026-10-01: The most slots auto considers: the widest rung of the decode batch ladder (traits::DECODE_BATCH_LADDER), the widest batch a decode step is padded to and captured at.

balance_slots: 2026-10-01: The slot count in 1..=max_slots that admits the most sequences at blocks_per_seq blocks each, kv_blocks_at(slots) giving the pool size for a count. On a tie the smaller count wins, so the chosen count never exceeds what its pool holds at full length (a count above it ties with the count one below, whose pool is at least as large) and the boot’s overcommit check passes. An error when max_slots is 0, when kv_blocks_at fails for every count, or when no count’s pool holds one full-length sequence.
build_model
loader_for_config: 2026-09-25: The weight loader for config.model_type, matched after lower-casing it and turning - and . into _. An unknown type is an error.