Module timing

Module timing 

Source
Expand description

2026-09-25: L1 GPU spans: CUDA event pairs around each serve lane on every step, and around each eager kernel launch on the steps the telemetry hub arms (one in kernel_span_every), at telemetry level Kernel only.

The bookkeeping is metrale_telemetry::kernel::SpanRing; this module is the CUDA half: the event pool (cuda_events) and the process-wide recorder that MetraleCudaBackend::new installs.

Lane spans are recorded on the registry’s stream (MetraleRegistry::raw_stream), which cudarc creates non-blocking, so a lane pair times the work queued on that stream only.

Owner: gpu-runtime (telemetry). Invariants:

  • No span synchronises: a completed pair is harvested with a non-blocking cuEventQuery, and a pair still in flight is left for a later harvest.
  • Below level Kernel no entry point reaches CUDA: each first reads the hub’s level or armed flag (a relaxed atomic load), and kernel_end acts only on a span kernel_begin opened.
  • No kernel span is opened on a stream that is being captured.

Structs§

GpuSpans
2026-09-25: The recorder install_for_stream installs; a later install replaces it.
KernelSpan
2026-09-25: An open kernel span.

Constants§

SPAN_SLOTS
2026-09-25: Event pairs in the pool. The ring hands out slots in turn; a span whose slot is still in flight is dropped and counted (spans_dropped).

Functions§

harvest
2026-09-25: Land every completed span. The scheduler calls it at each step begin.
install_for_stream
2026-09-25: Build and install the recorder for a backend whose stream is stream, when telemetry is at level Kernel. Called from MetraleCudaBackend::new after the registry loads. A pool that cannot be created leaves L1 off and logs why.
kernel_begin
2026-09-25: A kernel span, when this step is armed for kernel spans and stream is not being captured (an event recorded into a capture would be a graph node, not a timestamp).
kernel_end
2026-09-25: Close a kernel span opened by kernel_begin.
lane_begin
2026-09-25: Open lane lane’s span; an index outside LANES is ignored.
lane_end
2026-09-25: Close lane lane’s span, if one is open.