Crate metrale_model_arch 

2026-09-25: Per-family model architectures, their weight loaders (weight_loader, mistral_loader) and the DFlash and MTP draft heads.

Owner: model-arch. Invariants: none beyond the types.

Modules

attn_v41

2026-09-25: DeepSeek-V4.1 Flash attention on the GPU: one forward for every layer.

deepseek_v4_mtp

2026-09-25: DeepSeek-V4-Flash multi-token-prediction (MTP) draft proposer.

deepseek_v41_layer

2026-09-25: DeepSeek-V4.1 Flash: one transformer block on the generic model.

deepseek_v41_ref

2026-09-25: DeepSeek-V4.1 Flash tiny-graph reference: the V4.1 graph on the CPU, checked against ds41_tiny_golden.json.

dflash_head

2026-09-25: DFlash block-diffusion draft head (arXiv 2602.06036), implementing DraftProposer.

engram_v41

2026-09-25: DeepSeek-V4.1 Flash engram on the GPU: n-gram hash, row gather, wkv projection and gate.

glm5next_dsa

2026-09-25: GLM-5.3-Flash DSA (DeepSeek Sparse Attention): the kernel handles, the per-layer geometry and its validation.

glm5next_dsa_ref

2026-09-25: CPU reference for the GLM-5.3-Flash DSA kpool indexer and NoPE MLA.

glm5next_kda

2026-09-25: The GLM-5.3-Flash KDA attention block: one bound self_attn block per KDA layer.

glm5next_kda_ref

2026-09-25: CPU reference for GLM-5.3-Flash KDA (Kimi Delta Attention).

glm5next_layer

2026-09-25: Glm5NextLayer, the GLM-5.3 decoder layer behind TransformerLayer: a KDA or DSA mixer, a dense or routed-MoE MLP, and the mHC hyper-connection.

glm5next_mhc

2026-09-25: Host launchers for the GLM-5.3 mHC kernels: expand, pre (the hc_mix + hc_finish pair), post, and the head mean.

glm5next_mlp

2026-09-25: GLM-5.3-Flash MLP: the dense SwiGLU FFN and the routed NVFP4 MoE, their kernels and per-rank geometry.

glm5next_mtp_head

2026-09-25: Glm5NextMtpHead: GLM-5.3’s MTP block as a DraftProposer, one draft token per forward_one.

glm5next_skeleton

2026-09-25: GLM-5.3-Flash text-model skeleton: topology, residual/norm/mHC wiring, structural weight contract and state plan, as data.

kimi_k3

2026-09-25: Kimi K3 in the engine: the bound decoder layer, its per-sequence state, and the CUDA launches for the KDA and MLA mixers, the packed LatentMoE experts and the dense MLP.

mistral_loader

2026-09-25: Weight loader for Mistral Small 4 (model_type = "mistral"): MLA attention with MoE FFNs.

moe_v41

2026-09-25: DeepSeek-V4.1 Flash MoE on streamed, still-quantized experts.

nemotron_mamba2

2026-09-25: Nemotron-H Mamba-2 SSM layer (no FFN), implementing TransformerLayer.

nemotron_moe

2026-09-25: Nemotron-H standalone MoE FFN layer, in two shapes:

precision_schedule

2026-09-25: A per-(layer, role) weight precision schedule: Role tags a tensor, Dtype is a requested precision, and PrecisionSchedule resolves a (layer, role) pair to a dtype.

seq_state_reserve

2026-09-25: Per-sequence device state the serve must reserve, computed from the config before the model exists.

tp_shard

2026-09-25: Tensor-parallel weight sharding helpers.

weight_loader

2026-09-25: The ModelWeightLoader trait, and one loader per architecture that turns a flat WeightStore into typed TransformerLayers.