Module glm5next_layer 

2026-09-25: Glm5NextLayer, the GLM-5.3 decoder layer behind TransformerLayer: a KDA or DSA mixer, a dense or routed-MoE MLP, and the mHC hyper-connection.

Owner: model-arch (GLM-5.3). Invariants:

• Every norm launches rms_norm_vanilla (x * w / rms), never rms_norm, which scales by 1 + w.
• Highway slots are per token: decode uses slot 0, prefill gives prompt token t slot t, and decode_batched gives verify row t slot t.
• The last text layer collapses the highway with hc_head_mean, which takes no weights.
• Any all-reduce of a mixer or MLP output happens before hc_post folds that output into the highway.

One mHC site, as forward_one and forward_k run it:

layer 0 only:  hc_expand(hidden) -> streams            [hc_mult, hidden] FP32
each site:     hc_pre(streams) -> y (into hidden), post, comb
               rms_norm_vanilla(y, site norm) -> normed
               sublayer(normed) -> out
               hc_post(out, residual = streams, post, comb) -> streams
last layer:    hc_head_mean(streams) -> hidden

hc_pre only reads streams, so hc_post takes the same buffer as its residual and writes its result over it.

Re-exports

pub use state::alloc_kda_ssm_state;

Modules

profile

2026-09-25: Per-section decode timing for the GLM-5.3 layers, enabled by METRALE_GLM_PROFILE.

state

2026-09-25: Pool-free allocation of a GLM-5.3 KDA layer’s per-sequence state.

Structs

Glm5NextLayer

2026-09-25: One bound GLM-5.3 decoder layer.

Glm5NextMhc

2026-09-25: The layer’s hyper-connection: the kernels, each site’s weights, and the config’s hc_mult, hc_sinkhorn_iters and hc_eps.

Enums

Glm5NextMixer

2026-09-25: The layer’s mixer. KDA’s decode is an inherent method that takes a KdaSeqState and a workspace and leaves its output in ws.final_out; DSA’s decode is the TransformerLayer method and writes its output over the buffer it was given. Glm5NextLayer::mixer_forward handles both.

Glm5NextMlpSite

2026-09-25: The layer’s MLP: dense for the layers the config lists in mlp_only_layers, routed MoE for the rest.

Functions

prefill_rows

2026-09-25: PREFILL_ROWS, overridable at launch with METRALE_GLM_PREFILL_ROWS (values below 1 or unparsable are ignored). 1 selects the per-token walk: Glm5NextLayer::prefill takes the batched sub-chunk path only when rows > 1. A width above DENSE_GEMV_BATCHM_MAX_M changes the projection numerics as well as the speed.