Crate metrale_model_engine 

2026-09-25: The model engine: the Model trait (traits), the transformer model (model), the single-request generate loop (engine), the model factory (factory), the Kimi K3 weight loader (kimi_k3_loader), the prefill chunk planner (prefill_plan) and the startup rank-agreement check (rank_agree).

Owner: model-engine. Invariants: none beyond the types.

Modules

engine

2026-09-25: A greedy generate loop for one request over the Model trait: prefill, then decode, taking each token with argmax_on_device.

factory

2026-09-25: Model factory: loader_for_config picks the ModelWeightLoader for a config’s model_type, and build_model builds the model from the config and the weights.

kimi_k3_loader

2026-09-25: The Kimi K3 ModelWeightLoader: binds BF16 or FP32 checkpoints, and packed MXFP4 experts only when K3_ALLOW_MXFP4=1.

model

2026-09-25: The generic transformer model, TransformerModel, and the served NLLB model.

prefill_plan

2026-09-27: Where prefill chunks end, and which chunk ends carry an SSM snapshot.

rank_agree

2026-09-25: Startup check that every rank holds the same values for the env-read scalars that shape the collective schedule.

traits

2026-09-25: The Model trait, the per-sequence state it works on (SequenceState), and the input and result types of its forward passes.