Expand description
2026-09-26: Low-level bindings: the RDMA verbs C shim with its rail
handshake and wire codecs, NCCL (feature nccl), NVML and cuFile loaded
with dlopen, and NVTX ranges (feature nvtx).
verbs and railset are compiled only under cfg(metrale_rdma_verbs),
which build.rs emits when it compiles the shim. env, handshake and
wire are always compiled, so their tests run without rdma-core.
Owner: metrale-gpu-sys. Invariants: none beyond the types.
Re-exports§
pub use wire::CacheServerParams;pub use wire::MODE_TCP;pub use wire::MODE_VERBS;pub use wire::RemoteQp;pub use wire::STATUS_ERR;pub use wire::STATUS_OK;pub use wire::VerbsClientParams;pub use wire::VerbsServerParams;
Modules§
- cufile
- 2026-09-26: cuFile (GPUDirect Storage) entry points, resolved from
libcufile with
dlopen, so a binary starts on a host without the library andCuFile::loadreports the failure instead. metrale-storage’s GDS probe and bench call them; docs/adr/0008-nvme-high-speed-swap.md records why the swap path does not use GDS on GB10. - env
- 2026-09-26: Environment lookups over a key list the caller passes: the RDMA clients read their device names, GID indices and depths through these. No key or fallback chain is fixed here.
- handshake
- 2026-09-26: The client side of the rail handshake as byte functions over
Read/Writethat take QP identities as(qpn, psn, gid)tuples, so they need noVerbs.railset::RailSetcalls them; tests/transcript_golden.rs drives them against a scripted peer. - nccl
- 2026-09-26: Raw NCCL bindings used by metrale-comm, plus the CUDA driver
stream and event calls it needs. The function declarations follow the
installed nccl.h (2.29.7);
NcclConfigdoes not (see its doc). - nvml
- 2026-09-26: NVML (the NVIDIA management library), resolved with
dlopen. - wire
- 2026-09-26: RDMA handshake wire codecs, used by the RDMA clients and re-exported by the storage peer daemons, so both ends share one codec.
Functions§
- verbs_
enabled - 2026-09-26: Whether build.rs compiled the verbs shim, i.e. whether
cfg(metrale_rdma_verbs)is set. Always compiled, so tests can assert it (tests/verbs_probe.rs, metrale-storage’s rdma_verbs_probe_tests.rs).