Crate metrale_model_layers 

2026-09-25: Model layers: the per-layer forward implementations, LoRA adapters, weight maps, the SSM reserve, speculative helpers and the vision/video preprocessing, plus the per-model-type dispatch predicates below.

Owner: model-layers. Invariants: none beyond the types.

Re-exports

pub use crate::mtp_carry::DEFAULT_MARCONI_MIN_TOKENS;

pub use crate::mtp_carry::set_marconi_min_tokens;

pub use vision_item::VisionItem;

Modules

circuit_exec

2026-09-28: The circuit executor: run a model’s decode from its fused circuit plan instead of the hand-written layer loops. At boot it instantiates the model’s circuit, fuses it under the live policy with the kernels the loaded modules contain, lays its buffers out in one workspace and compiles a straight-line program::Program for single-sequence decode and one per multi-sequence batch width; a decode step then runs the program for its width between the legacy prologue (embedding, metadata upload) and the logits.

drafter_context

2026-09-25: Resolution of the MTP drafter context from the environment: whether the drafter KV is batch-prefilled over the prompt (prefill) and whether it is carried across the turns of a session (carry).

forward

2026-09-25: Per-layer decode forwards written against GpuBackend and the QuantWeights trait, so a driver runs a layer without knowing the weight format.

layer

2026-09-25: Types every TransformerLayer shares: per-sequence layer state, the per-pass ForwardContext, and the attention and GDN metadata a pass hands its layers.

layers

2026-09-25: The layer types (FfnComponent and the modules below) and the kernel-handle resolvers several layers share.

lora

2026-09-25: PEFT LoRA adapters: audit, pack into the rank-padded slot pool and the expert pool, slot routing and swap, and token overlays. “LoRA” here means adapters; kv_lora_rank / q_lora_rank in the model config are MLA dimensions, unrelated.

mtp_carry

2026-09-25: Carry the MTP drafter’s KV from one turn of a session to the next, so a warm turn adopts the previous turn’s drafter rows and appends only the new span, plus the SSM snapshot restore threshold.

quant_format

2026-09-25: The quantized-weight layout of a checkpoint, and its ignore list: CompressedTensorsFormat, ModeloptFormat or Fp8BlockScaledFormat, chosen by detect_quant_format.

speculative

2026-09-25: The draft-proposer interface for speculative decoding (DraftProposer, ProposerState), and the MTP knobs, ladder, tree shape and verify keys in its submodules.

ssm_reserve

2026-09-25: GPU reserve terms for SSM (linear-attention) state: verify-pool slot counts and capacities, pool bytes, Marconi snapshot slots, and the decode-rollback ring depth (decode_ring).

video_decode_ffmpeg

2026-09-25: Decodes video containers to RGB frames by running ffmpeg as a subprocess, for every container that is not a GIF.

video_preprocess

2026-09-25: Video to patch tensor, the temporal counterpart of crate::vision_preprocess.

vision_item

2026-09-25: The unit of vision input handed to the model.

vision_preprocess

2026-09-25: CPU-side image preprocessing for the vision encoder. Decodes a base64 image, resizes it onto a grid of patch_size × spatial_merge_size cells, normalises it with the checkpoint’s mean and std, and lays the patches out as a flat f32 tensor.

vision_preprocess_glm

2026-09-25: GLM image geometry: smart_resize to a canvas aligned to patch_size × spatial_merge_size, an aspect-preserving fit that never upscales, a black pad to the canvas, and the 2×2-block-major patch order. vision_preprocess uses it when VisionConfig::block_major_patches is set.

weight_map

2026-09-25: Maps HuggingFace safetensors tensor names onto the typed per-layer weight structures, through the weight_map/ sub-modules re-exported here.

Functions

model_type_ships_vanilla_norm_weights

2026-09-25: The predicate behind ships_vanilla_norm_weights, on the bare model_type, so a test needs no ModelConfig.

requires_single_chunk_prefill

2026-09-25: Must chunked prefill run as one chunk for this model?

ships_vanilla_norm_weights

2026-09-26: True when the checkpoint’s RMSNorm weights are plain multipliers, out = x * w / rms (the rms_norm_vanilla kernel), rather than offsets from 1, out = x * (1 + w) / rms (the rms_norm kernel). It selects the final-norm kernel (model-engine model/impl_a1.rs) and the attention norm kernels (layers/qwen3_attention/init_proj_kernels.rs and init_prefill_kernels.rs). The answer comes from model_type alone, not from the weight values.