Module profile

Expand description

2026-09-25: Per-section decode timing for the GLM-5.3 layers, enabled by METRALE_GLM_PROFILE.

Owner: model-arch (GLM-5.3). Invariants:

Unless METRALE_GLM_PROFILE is 1 or 2, start and start_hot return None and no span synchronises.
end and end_us synchronise the stream before reading the clock, so a profiled run is serialised and its throughput is not a serving figure.

Constants§

DSA_ATTEND
DSA_INDEXER
DSA_PROJ
DSA_SELECT
KDA
KDA_BACK
KDA_FRONT: 2026-09-25: Sub-spans of the KDA mixer: the projections before the recurrence, the recurrence, and the output side, timed in glm5next_kda’s decode and prefill.
KDA_RECUR
MHC
MHC_POST: 2026-09-25: hc_post, plus hc_head_mean on the last layer; MHC holds hc_expand and hc_pre.
MLP_DENSE
MOE_COMBINE
MOE_EXPERTS
MOE_HOSTSYNC
MOE_ROUTER
MOE_SHARED
NORM
REDUCE_ATTN
REDUCE_ATTN_BAR: 2026-09-25: A 2-byte all-reduce issued just before the real one while profiling (Glm5NextLayer::reduce_probe), so that the ranks’ arrival skew is charged here and not to REDUCE_ATTN or REDUCE_MLP.
REDUCE_ATTN_ENQ: 2026-09-25: Host time to enqueue the attention and MLP all-reduces, without a synchronise. REDUCE_ATTN and REDUCE_MLP then time only the synchronize that follows the enqueue.
REDUCE_MLP
REDUCE_MLP_BAR
REDUCE_MLP_ENQ

end
end_nosync: 2026-09-25: Record a span without synchronising: host wall time only.
end_us: 2026-09-25: end, returning the span in microseconds; 0.0 when t0 is None.
full: 2026-09-25: True only at level 1, the per-kernel spans.
on
probe_buf: 2026-09-25: 4-byte device buffer for the rendezvous probe, allocated on first use; 0 when the allocation failed, and reduce_probe then skips the probe.
start: 2026-09-25: Open a span at level 1; None otherwise, which makes end a no-op.
start_hot: 2026-09-25: Open a span at level 1 or 2, for the collectives and their rendezvous probe.
stash_route: 2026-09-25: Called from forward_moe after the ids are read back. No-op unless tracing.
step: 2026-09-25: Count one token. Every 8th token, log the cumulative time per token and calls per token of each bucket.
trace_bar: 2026-09-25: Log the trace line of one reduce site: site is attn or mlp, and moe says whether this layer’s MLP is routed. No-op unless tracing.
trace_on: 2026-09-25: METRALE_GLM_ROUTE_TRACE=1: trace_bar logs one line per reduce site per layer with the rendezvous time and the expert ids stash_route last saved. Read once. It logs only while profiling is on too, because trace_bar is called from reduce_probe.