Expand description
2026-09-25: Typed weight binding for the GLM-5.3-Flash KDA attention blocks.
Owner: model-arch (GLM-5.3-Flash KDA). Invariants:
bind_kda_weightsreturns weights only when a layer’sself_attn.*tensors are exactlyKDA_TENSORS, each with its listed dtype and shape; anything else is an error.- Bytes are uploaded unchanged: no cast, requantisation or dequantisation.
The MTP layer is DSA-shaped (it loads through glm5next_dsa, see
weight_loader/glm5_next_mtp.rs) and is not bindable here; classify_attn_block tells the
two apart from the tensor names alone.
The checkpoint stores three rank-3 conv tensors ([qkv, 1, kernel]). The binder concatenates
them in q, k, v order and uses the result as [conv_dim, kernel]: dropping the singleton
dimension moves no bytes. A_log (per head) and dt_bias (per channel) are F32; a BF16 copy
of either is refused.
Structs§
- KdaBind
Report - 2026-09-25: Per-layer counts from
bind_kda_weights. Tensors outsideself_attn.are counted innon_attn_seenand never bound. - RawTensor
- 2026-09-25: One tensor as it sits in the checkpoint: dtype, shape and raw little-endian bytes.
- Tensor
Spec - 2026-09-25: One entry of
KDA_TENSORS: aself_attntensor’s name, dtype and shape.
Enums§
- Attn
Block Kind - 2026-09-25: What kind of attention block a layer’s tensor names describe.
- KdaDtype
- 2026-09-25: The dtypes the KDA binder accepts.
Constants§
- DSA_
MARKERS - 2026-09-25: Names that identify a DSA (
deepseek_sparse_attention) block, the MTP layer included, so a caller can classify a layer without its index. - KDA_
TENSORS
Traits§
- KdaTensor
Source - 2026-09-25: A checkpoint slice scoped to one decoder layer. Names are layer-relative
(
self_attn.q_proj.weight), so the same binder works for any layer index.
Functions§
- bind_
kda_ weights - 2026-09-25: Bind one KDA block: exact tensor set, dtypes, shapes and byte counts, or an error.
- classify_
attn_ block - 2026-09-25: Classify from tensor names alone:
Dsawhen everyDSA_MARKERSname is present, elseKdawhen everyKDA_TENSORSname is present, elseUnknown.