Expand description
2026-09-25: L1 GPU spans: CUDA event pairs around each serve lane on every
step, and around each eager kernel launch on the steps the telemetry hub arms
(one in kernel_span_every), at telemetry level Kernel only.
The bookkeeping is metrale_telemetry::kernel::SpanRing; this module is the
CUDA half: the event pool (cuda_events) and the process-wide recorder that
MetraleCudaBackend::new installs.
Lane spans are recorded on the registry’s stream (MetraleRegistry::raw_stream),
which cudarc creates non-blocking, so a lane pair times the work queued on that
stream only.
Owner: gpu-runtime (telemetry). Invariants:
- No span synchronises: a completed pair is harvested with a non-blocking
cuEventQuery, and a pair still in flight is left for a later harvest. - Below level
Kernelno entry point reaches CUDA: each first reads the hub’s level or armed flag (a relaxed atomic load), andkernel_endacts only on a spankernel_beginopened. - No kernel span is opened on a stream that is being captured.
Structs§
- GpuSpans
- 2026-09-25: The recorder
install_for_streaminstalls; a later install replaces it. - Kernel
Span - 2026-09-25: An open kernel span.
Constants§
- SPAN_
SLOTS - 2026-09-25: Event pairs in the pool. The ring hands out slots in turn; a span
whose slot is still in flight is dropped and counted (
spans_dropped).
Functions§
- harvest
- 2026-09-25: Land every completed span. The scheduler calls it at each step begin.
- install_
for_ stream - 2026-09-25: Build and install the recorder for a backend whose stream is
stream, when telemetry is at levelKernel. Called fromMetraleCudaBackend::newafter the registry loads. A pool that cannot be created leaves L1 off and logs why. - kernel_
begin - 2026-09-25: A kernel span, when this step is armed for kernel spans and
streamis not being captured (an event recorded into a capture would be a graph node, not a timestamp). - kernel_
end - 2026-09-25: Close a kernel span opened by
kernel_begin. - lane_
begin - 2026-09-25: Open lane
lane’s span; an index outsideLANESis ignored. - lane_
end - 2026-09-25: Close lane
lane’s span, if one is open.