Module glm5next_skeleton

Module glm5next_skeleton 

Source
Expand description

2026-09-25: GLM-5.3-Flash text-model skeleton: topology, residual/norm/mHC wiring, structural weight contract and state plan, as data.

Owner: model-arch (GLM-5.3). Invariants:

  • A skeleton from from_config has one layers entry per layer_types entry, each with hyper_connection set and is_mtp clear. The MTP layer, when present, is held in mtp, at index num_hidden_layers, with no hyper-connection.

It executes nothing: there is no forward pass here, and the MLP site is a named step (ResidualStep::Mlp) with its kind recorded.

Facts about the reference checkpoint LibertAIDAI/GLM-5.3-Flash-NVFP4@9e0d74e3, checked against its fixtures by crates/model-engine/tests/glm5next_skeleton.rs:

  • The structural (non-MLP) surface is 1,047 tensors in three signatures: 34 KDA layers x 23, 11 DSA layers x 22, the MTP layer x 20, plus 3 non-layer tensors.
  • The MTP layer has no hyper-connection: all 270 hc_* tensors are on layers 0..=44.
  • The final collapse is an unweighted mean: the checkpoint has no hc_head tensor.
  • Layers 0..=2 carry a dense MLP (first_k_dense_replace = 3); the other 42 text layers and the MTP layer route to experts.

Structs§

Glm5NextTextSkeleton
SkeletonLayer
2026-09-25: One decoder layer’s structure.
StateBudget
2026-09-25: Per-sequence state bytes, built by Glm5NextTextSkeleton::state_budget from the config’s geometry. per_rank divides only the KDA state and the DSA KV.
StructuralAccounting

Enums§

FinalStep
2026-09-25: What the model does after the last text layer.
Mixer
2026-09-25: Which mixer a layer runs. Narrower than LayerType: from_config refuses the kinds GLM-5.3 does not have rather than carrying them as unreachable arms.
Mlp
2026-09-25: Which MLP a layer runs. Recorded, not executed, by this module.
ResidualStep
2026-09-25: One step of a layer’s residual path, in execution order.
Site
StateKind
2026-09-25: Which per-sequence state a layer needs. Each layer needs exactly one kind.