Expand description
2026-09-25: GLM-5.3-Flash text-model skeleton: topology, residual/norm/mHC wiring, structural weight contract and state plan, as data.
Owner: model-arch (GLM-5.3). Invariants:
- A skeleton from
from_confighas onelayersentry perlayer_typesentry, each withhyper_connectionset andis_mtpclear. The MTP layer, when present, is held inmtp, at indexnum_hidden_layers, with no hyper-connection.
It executes nothing: there is no forward pass here, and the MLP site is a named step
(ResidualStep::Mlp) with its kind recorded.
Facts about the reference checkpoint LibertAIDAI/GLM-5.3-Flash-NVFP4@9e0d74e3, checked
against its fixtures by crates/model-engine/tests/glm5next_skeleton.rs:
- The structural (non-MLP) surface is 1,047 tensors in three signatures: 34 KDA layers x 23, 11 DSA layers x 22, the MTP layer x 20, plus 3 non-layer tensors.
- The MTP layer has no hyper-connection: all 270
hc_*tensors are on layers 0..=44. - The final collapse is an unweighted mean: the checkpoint has no
hc_headtensor. - Layers 0..=2 carry a dense MLP (
first_k_dense_replace = 3); the other 42 text layers and the MTP layer route to experts.
Structs§
- Glm5
Next Text Skeleton - Skeleton
Layer - 2026-09-25: One decoder layer’s structure.
- State
Budget - 2026-09-25: Per-sequence state bytes, built by
Glm5NextTextSkeleton::state_budgetfrom the config’s geometry.per_rankdivides only the KDA state and the DSA KV. - Structural
Accounting
Enums§
- Final
Step - 2026-09-25: What the model does after the last text layer.
- Mixer
- 2026-09-25: Which mixer a layer runs. Narrower than
LayerType:from_configrefuses the kinds GLM-5.3 does not have rather than carrying them as unreachable arms. - Mlp
- 2026-09-25: Which MLP a layer runs. Recorded, not executed, by this module.
- Residual
Step - 2026-09-25: One step of a layer’s residual path, in execution order.
- Site
- State
Kind - 2026-09-25: Which per-sequence state a layer needs. Each layer needs exactly one kind.