Trait CommBackend 

pub trait CommBackend: Send + Sync {
    // Required methods
    fn all_reduce(&self, ptr: u64, bytes: usize) -> Result<()>;
    fn all_gather(
        &self,
        send_ptr: u64,
        recv_ptr: u64,
        bytes: usize,
    ) -> Result<()>;
    fn reduce_scatter(
        &self,
        send_ptr: u64,
        recv_ptr: u64,
        bytes: usize,
    ) -> Result<()>;
    fn broadcast(&self, ptr: u64, bytes: usize, root: usize) -> Result<()>;
    fn barrier(&self) -> Result<()>;
    fn send_to(
        &self,
        ptr: u64,
        bytes: usize,
        dest_rank: usize,
        stream: u64,
    ) -> Result<()>;
    fn recv_from(
        &self,
        ptr: u64,
        bytes: usize,
        src_rank: usize,
        stream: u64,
    ) -> Result<()>;
    fn rank(&self) -> usize;
    fn world_size(&self) -> usize;

    // Provided methods
    fn recv_command_u32(&self, ptr: u64, root: usize) -> Result<()> { ... }
    fn all_reduce_async(
        &self,
        ptr: u64,
        bytes: usize,
        compute_stream: u64,
    ) -> Result<()> { ... }
    fn register_buffer(&self, _ptr: u64, _bytes: usize) -> Result<u64> { ... }
    fn deregister_buffer(&self, _handle: u64) -> Result<()> { ... }
    fn symmetric_alloc(&self, _bytes: usize) -> Result<u64> { ... }
    fn symmetric_free(&self, _ptr: u64) -> Result<()> { ... }
    fn set_add_kernel(&self, _handle: u64) { ... }
    fn group_start(&self) -> Result<()> { ... }
    fn group_end(&self) -> Result<()> { ... }
    fn is_healthy(&self) -> bool { ... }
    fn attempt_reconnect(&self) -> Result<()> { ... }
}

2026-09-26: Collective and point-to-point operations on raw device pointers and byte counts. A pointer is a u64 (a CUDA CUdeviceptr), so this crate does not depend on metrale-gpu-runtime.

Required Methods

fn all_reduce(&self, ptr: u64, bytes: usize) -> Result<()>

2026-09-26: Sum bytes at ptr across all ranks, in place; every rank gets the result. NcclBackend reduces BF16 elements.

fn all_gather(&self, send_ptr: u64, recv_ptr: u64, bytes: usize) -> Result<()>

2026-09-26: Each rank contributes bytes from send_ptr; every rank receives all contributions, in rank order, at recv_ptr.

fn reduce_scatter( &self, send_ptr: u64, recv_ptr: u64, bytes: usize, ) -> Result<()>

2026-09-26: Sum across ranks and give each rank its bytes-sized share at recv_ptr. NcclBackend sums u8 elements.

fn broadcast(&self, ptr: u64, bytes: usize, root: usize) -> Result<()>

2026-09-26: Copy bytes at ptr from rank root to every rank, in place.

fn barrier(&self) -> Result<()>

2026-09-26: A barrier across ranks. NcclBackend enqueues a zero-count all-reduce on its stream and returns without waiting for it.

fn send_to( &self, ptr: u64, bytes: usize, dest_rank: usize, stream: u64, ) -> Result<()>

2026-09-26: Send bytes at ptr to dest_rank, enqueued on stream. Pairs with a recv_from of the same size on dest_rank.

fn recv_from( &self, ptr: u64, bytes: usize, src_rank: usize, stream: u64, ) -> Result<()>

2026-09-26: Receive bytes into ptr from src_rank, enqueued on stream. Pairs with a send_to of the same size on src_rank.

fn rank(&self) -> usize

2026-09-26: This rank’s index, from 0.

fn world_size(&self) -> usize

2026-09-26: Number of ranks.

Provided Methods

fn recv_command_u32(&self, ptr: u64, root: usize) -> Result<()>

2026-09-26: On a non-root rank, receive the first u32 of the next worker command from root, the receive side of a 4-byte broadcast. Unlike broadcast, it may wait without a deadline, so an implementation must keep checking for transport errors while it waits. The command’s later words use broadcast. The default refuses root and otherwise calls broadcast(ptr, 4, root).

fn all_reduce_async( &self, ptr: u64, bytes: usize, compute_stream: u64, ) -> Result<()>

2026-09-26: All-reduce ordered against compute_stream on the device: the reduce starts after the work already queued there and later work there waits for it. The host does not wait. The default ignores compute_stream and calls all_reduce.

fn register_buffer(&self, _ptr: u64, _bytes: usize) -> Result<u64>

2026-09-26: Register a device buffer with the backend and return an opaque handle (ncclCommRegister in NcclBackend). The default registers nothing and returns 0.

fn deregister_buffer(&self, _handle: u64) -> Result<()>

2026-09-26: Deregister a handle from register_buffer.

fn symmetric_alloc(&self, _bytes: usize) -> Result<u64>

2026-09-26: Allocate bytes of device memory from the backend’s allocator (ncclMemAlloc in NcclBackend) and return the pointer. The default returns Err.

fn symmetric_free(&self, _ptr: u64) -> Result<()>

2026-09-26: Free a pointer from symmetric_alloc. The default returns Err.

fn set_add_kernel(&self, _handle: u64)

2026-09-26: Hand the backend the bf16_add_inplace kernel handle. The model engine looks it up and passes it after registering its buffers. NcclBackend takes its 2-rank send/recv all-reduce path only once it is set.

fn group_start(&self) -> Result<()>

2026-09-26: Open a group: NcclBackend calls ncclGroupStart, so the operations up to group_end are submitted together. The default does nothing.

fn group_end(&self) -> Result<()>

2026-09-26: Close the group opened by group_start.

fn is_healthy(&self) -> bool

2026-09-26: false after the backend has seen a transport error or a completion timeout, until a successful attempt_reconnect. NcclBackend also queries ncclCommGetAsyncError. The default returns true.

fn attempt_reconnect(&self) -> Result<()>

2026-09-26: Replace a failed communicator. NcclBackend aborts it and repeats the TCP bootstrap, so every rank must call this at the same time. The default does nothing.

Implementors

impl CommBackend for NcclBackend

impl CommBackend for SingleGpuBackend