Module expert_peer 

2026-09-25: The expert peer, a process that serves an expert store’s records to RdmaTier clients, and its wire format (little-endian):

1. On accept the peer sends the manifest, [u32 len][len bytes of JSON].
2. The client sends one transport byte, MODE_TCP or MODE_VERBS.
3. TCP: the client sends [u32 layer][u32 expert]; the peer answers [STATUS_OK][record_stride bytes] or [STATUS_ERR], until the client hangs up or sends SHUTDOWN_MARKER in both fields. Verbs: the peer maps and registers every layer file for remote read, publishes its rails, and waits for hangup while the client reads records itself.

The peer uses no CUDA.

Owner: metrale-storage peers. Invariants:

• A verbs connection reserves index.total_bytes() in the ledger before it maps or registers anything.

Structs

RdmaConfig

2026-09-25: Settings of the verbs transport, unused by TCP clients. A client asking for n rails gets the first n (device, gid_idx) entries; the peer registers every layer file on each.

VerbsClientParams

2026-09-26: The client’s half of the verbs handshake: its QP identity only.

VerbsServerParams

2026-09-26: The server’s half of the read-only verbs handshake (expert, weight and LoRA tiers): its QP identity plus one registered MR per MoE layer (expert tier) or per shard (weight and LoRA tiers). The expert tier reads at layers[layer].0 + expert * record_stride.

Constants

MODE_TCP

2026-09-26: Transport byte for two-sided TCP record streaming from the expert peer.

MODE_VERBS

2026-09-26: Transport byte for one-sided RDMA READ: the server sends its MRs as VerbsServerParams and the client reads records from them.

SHUTDOWN_MARKER

2026-09-25: In both fields of a TCP request, asks the peer to close the connection.

STATUS_ERR

STATUS_OK

Functions

decode_request

2026-09-25: Decode a TCP request; the inverse of encode_request.

encode_request

2026-09-25: Encode a TCP request, [u32 layer][u32 expert] little-endian.

read_manifest

2026-09-25: Read and parse the length-prefixed manifest at the start of a connection (used by expert_tier_rdma). Fails on a length of 0 or above 16 MiB.

read_server_rails

2026-09-26: Read want rails of VerbsServerParams framed by a leading [u8 n_rails]. Fails when the framed count is 0, above 8, or not want.

serve

2026-09-25: Serve the store in dir on addr, one thread per connection. Accept errors are logged and skipped; only opening the store or binding returns an error.

write_server_rails

2026-09-26: Write [u8 n_rails] followed by each rail’s VerbsServerParams. Fails before writing anything when the count is 0 or above 8.