Expand description
2026-09-25: Per-sequence device state the serve must reserve, computed from the config before the model exists.
The server’s preflight_reserve adds this charge (per_sequence_reserve)
before the model is built, when there is no layer list, proposer or
GpuBackend, so it is a function of the config alone, like the
ssm_reserve terms.
Charged, per sequence:
- the target stack’s DSA indexer caches (
Glm5NextDsaState), one per text DSA layer; - the draft proposer’s own state (what
Glm5NextMtpHead’salloc_stateallocates), in its own field: the proposer is not aTransformerLayerand is not in the layer list.
Not charged here:
- KDA recurrent and conv state: pool-owned (
ssm_reserve::PoolPlan);meta.rsgives pool-backed linear-attention layers pool addresses instead of callingalloc_state; - the paged KV pool: this reserve is subtracted from the KV budget.
The result is per-rank bytes; nothing here divides by a world size.
Owner: model-arch (serve memory reserve). Invariants:
- Any
model_typeother thanglm5_nextis charged zero.
Structs§
- PerSequence
State - 2026-09-25: Per-sequence, per-rank device state, split by owner.
Functions§
- per_
sequence_ state_ bytes - 2026-09-25: Per-sequence owned device state for
config, at a context ofmax_seq_lentokens.