ModelEp

metrale_model_engine::traits

Trait ModelEp 

Source 
pub trait ModelEp {
    // Provided methods
    fn ep_worker_step(
        &self,
        _slots: &mut [Option<SequenceState>],
    ) -> Result<bool> { ... }
    fn is_ep(&self) -> bool { ... }
    fn ep_broadcast_cmd(&self, _cmd: u32) -> Result<()> { ... }
    fn ep_broadcast_cmd_for_seq(&self, _seq_id: u32, _cmd: u32) -> Result<()> { ... }
    fn ep_protocol_v2(&self) -> bool { ... }
    fn ep_broadcast_tokens(&self, _tokens: &[u32]) -> Result<Vec<u32>> { ... }
}
Expand description

2026-09-26: The multi-rank (EP) command protocol.

Provided Methods§

Source

fn ep_worker_step(&self, _slots: &mut [Option<SequenceState>]) -> Result<bool>

2026-09-25: EP worker step, on a rank above 0: receive a (seq_id, cmd) from rank 0 and run the command in slot seq_id. Returns Ok(false) when the worker should shut down.

An Err carrying EpCommandFailed means the command ran and failed, and the worker stays up; any other Err means the receive failed, and the worker exits.

slots is sized to --max-batch-size; a seq_id >= slots.len() fails. Default: Ok(true).

Source

fn is_ep(&self) -> bool

2026-09-25: Whether the model runs the multi-rank command protocol. The scheduler then turns off its single-GPU fused paths, the mixed forward among them. Default false.

Source

fn ep_broadcast_cmd(&self, _cmd: u32) -> Result<()>

2026-09-25: Send a command word to every worker rank. Default: Ok(()).

Source

fn ep_broadcast_cmd_for_seq(&self, _seq_id: u32, _cmd: u32) -> Result<()>

2026-09-25: Send a (seq_id, cmd) pair to every worker rank, as the first broadcast of a command; its follow-up broadcasts use Self::ep_broadcast_cmd. When Self::ep_protocol_v2 is false, seq_id is not sent. Default: Ok(()).

Source

fn ep_protocol_v2(&self) -> bool

2026-09-25: Whether the EP protocol carries a seq_id with each command (METRALE_EP_PROTOCOL=v2). Default false.

Source

fn ep_broadcast_tokens(&self, _tokens: &[u32]) -> Result<Vec<u32>>

2026-09-25: Broadcast a token array to every worker rank in one broadcast. Default: an empty Vec.

Implementors§