Struct NcclBackend 

pub struct NcclBackend { /* private fields */ }

2026-09-26: NCCL communicator plus the streams, events and receive buffer its operations use.

Implementations

impl NcclBackend

pub fn new( rank: usize, world_size: usize, master_addr: &str, master_port: u16, stream: u64, recv_capacity: usize, ) -> Result<Self>

2026-09-26: Bootstrap the communicator. Rank 0 binds 0.0.0.0:master_port and sends the unique id to world_size - 1 connections; other ranks connect to master_addr:master_port, retrying once a second for up to 600 attempts. recv_capacity is the largest 2-rank all-reduce payload in bytes (serve passes required_model_recv_bytes); it is used only when world_size == 2.

Errors

An invalid METRALE_COMM_DIAGNOSTICS value or rank/world pair, a failed bootstrap or NCCL init, a failed stream or event creation, a zero recv_capacity at world_size == 2, or a failed receive-buffer allocation. A failed registration of the receive buffer is only logged.

Trait Implementations

impl CommBackend for NcclBackend

fn all_reduce(&self, ptr: u64, bytes: usize) -> Result<()>

2026-09-26: Sum bytes at ptr across all ranks, in place; every rank gets the result. NcclBackend reduces BF16 elements.

fn all_reduce_async( &self, ptr: u64, bytes: usize, compute_stream: u64, ) -> Result<()>

2026-09-26: All-reduce ordered against compute_stream on the device: the reduce starts after the work already queued there and later work there waits for it. The host does not wait. The default ignores compute_stream and calls all_reduce.

fn register_buffer(&self, ptr: u64, bytes: usize) -> Result<u64>

2026-09-26: Register a device buffer with the backend and return an opaque handle (ncclCommRegister in NcclBackend). The default registers nothing and returns 0.

fn deregister_buffer(&self, handle: u64) -> Result<()>

2026-09-26: Deregister a handle from register_buffer.

fn symmetric_alloc(&self, bytes: usize) -> Result<u64>

2026-09-26: Allocate bytes of device memory from the backend’s allocator (ncclMemAlloc in NcclBackend) and return the pointer. The default returns Err.

fn symmetric_free(&self, ptr: u64) -> Result<()>

2026-09-26: Free a pointer from symmetric_alloc. The default returns Err.

fn set_add_kernel(&self, handle: u64)

2026-09-26: Hand the backend the bf16_add_inplace kernel handle. The model engine looks it up and passes it after registering its buffers. NcclBackend takes its 2-rank send/recv all-reduce path only once it is set.

fn all_gather(&self, send_ptr: u64, recv_ptr: u64, bytes: usize) -> Result<()>

2026-09-26: Each rank contributes bytes from send_ptr; every rank receives all contributions, in rank order, at recv_ptr.

fn reduce_scatter( &self, send_ptr: u64, recv_ptr: u64, bytes: usize, ) -> Result<()>

2026-09-26: Sum across ranks and give each rank its bytes-sized share at recv_ptr. NcclBackend sums u8 elements.

fn broadcast(&self, ptr: u64, bytes: usize, root: usize) -> Result<()>

2026-09-26: Copy bytes at ptr from rank root to every rank, in place.

fn recv_command_u32(&self, ptr: u64, root: usize) -> Result<()>

2026-09-26: On a non-root rank, receive the first u32 of the next worker command from root, the receive side of a 4-byte broadcast. Unlike broadcast, it may wait without a deadline, so an implementation must keep checking for transport errors while it waits. The command’s later words use broadcast. The default refuses root and otherwise calls broadcast(ptr, 4, root).

fn barrier(&self) -> Result<()>

2026-09-26: A barrier across ranks. NcclBackend enqueues a zero-count all-reduce on its stream and returns without waiting for it.

fn send_to( &self, ptr: u64, bytes: usize, dest_rank: usize, stream: u64, ) -> Result<()>

2026-09-26: Send bytes at ptr to dest_rank, enqueued on stream. Pairs with a recv_from of the same size on dest_rank.

fn recv_from( &self, ptr: u64, bytes: usize, src_rank: usize, stream: u64, ) -> Result<()>

2026-09-26: Receive bytes into ptr from src_rank, enqueued on stream. Pairs with a send_to of the same size on src_rank.

fn group_start(&self) -> Result<()>

2026-09-26: Open a group: NcclBackend calls ncclGroupStart, so the operations up to group_end are submitted together. The default does nothing.

fn group_end(&self) -> Result<()>

2026-09-26: Close the group opened by group_start.

fn is_healthy(&self) -> bool

2026-09-26: false after the backend has seen a transport error or a completion timeout, until a successful attempt_reconnect. NcclBackend also queries ncclCommGetAsyncError. The default returns true.

fn attempt_reconnect(&self) -> Result<()>

2026-09-26: Replace a failed communicator. NcclBackend aborts it and repeats the TCP bootstrap, so every rank must call this at the same time. The default does nothing.

fn rank(&self) -> usize

2026-09-26: This rank’s index, from 0.

fn world_size(&self) -> usize

2026-09-26: Number of ranks.

impl Drop for NcclBackend

fn drop(&mut self)

Executes the destructor for this type.

impl Send for NcclBackend

impl Sync for NcclBackend