Module weight_peer 

2026-09-26: The weight peer: a daemon that holds models’ safetensors shards mapped in its RAM and lets a client read any tensor with one-sided RDMA. The client is RdmaWeightLoader in model-weights (weight_tier_rdma.rs).

Protocol on one TCP connection, lengths little-endian (wire, serve):

1. The client sends the model request, [u32 len][len bytes of id or path].
2. The peer stages that model (once per directory) and sends the manifest, [u32 len][len bytes of JSON] (WeightManifest).
3. The client sends [u8 transport mode]; only MODE_VERBS is served.
4. The client sends [u8 n_rails]. The peer registers every shard mapping for remote read on each rail and sends one VerbsServerParams per rail, whose layers holds each shard’s (base, rkey). The client sends n_rails again and its QP parameters per rail; the peer connects, acknowledges, and waits for the client to close the connection while it reads the tensors.

Owner: storage (weight peer). Invariants: manifest and wire compile on every platform; serve and shard only on unix.

Structs

WeightManifest

2026-09-25: A staged model’s manifest, sent as length-prefixed JSON after the client’s model request (write_weight_manifest). The per-shard (base, rkey) pairs are sent later, in the verbs handshake.

WeightPeerConfig

2026-09-25: The peer’s rails, its staging ceiling, and which model directories a client may request.

WeightTensorRecord

2026-09-25: Where one tensor of a staged model lies in its shard file. The client reads [shard_base + offset_in_shard, + len).

Constants

MODEL_REQUEST_MAX

2026-09-25: The longest model request, in bytes, either side accepts; a longer length prefix is refused before any buffer is allocated.

Functions

rail_for_tensor

2026-09-25: The rail for tensor tensor_index: tensor_index % n_rails, with n_rails taken as at least 1. The client’s read loop calls it.

read_model_request

2026-09-25: Read a model request written by write_model_request.

read_weight_manifest

2026-09-25: Read and parse a manifest written by write_weight_manifest.

serve

2026-09-25: Stage cfg.staged_dirs, then accept connections on addr, one thread each, until the listener ends. A directory that fails to stage is logged and skipped.

tensor_remote_addr

2026-09-25: The peer address of a tensor’s first byte: the shard MR’s base plus offset_in_shard, which already counts the size prefix and header.

write_model_request

2026-09-25: Write the model request id.

write_weight_manifest

2026-09-25: Serialize m to JSON and write it with its length prefix.