Struct TpAttentionDims 

pub struct TpAttentionDims {
    pub tp_rank: usize,
    pub tp_size: usize,
    pub h: usize,
    pub head_dim: usize,
    pub full_q_n: usize,
    pub full_o_in: usize,
    pub full_kv_n: usize,
    pub gated: bool,
}

2026-09-25: Pre-TP-shard attention dimensions reconstructed from config.

The server’s serve_phases/topology.rs divides num_attention_heads and num_key_value_heads by the TP size before any loader runs, so config holds per-rank head counts. The full_* fields multiply back up to the pre-shard sizes the shard functions take.

When config.attn_gated is true the Q projection’s output includes the gate, doubling it: full_q_n includes the gate and full_o_in does not.

Fields

tp_rank: usize

tp_size: usize

2026-09-25: tp_world_size clamped to >= 1.

h: usize

2026-09-25: Hidden size, not divided per rank.

head_dim: usize

full_q_n: usize

2026-09-25: Q-projection output dim, doubled for gated attention.

full_o_in: usize

2026-09-25: O-projection input dim, the un-gated attention output: num_attention_heads * tp_size * head_dim, not doubled.

full_kv_n: usize

2026-09-25: num_key_value_heads_local * tp_size * head_dim: full K/V pre-shard.

gated: bool

2026-09-25: config.attn_gated.

Implementations

impl TpAttentionDims

pub fn from_config(config: &ModelConfig) -> Self

pub fn proj_shape(&self, name: &str) -> Option<(usize, usize, TpShardKind)>

2026-09-25: (out_dim, in_dim, kind) for a given QKVO projection; None for any other name.

Trait Implementations

impl Clone for TpAttentionDims

fn clone(&self) -> TpAttentionDims

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TpAttentionDims

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Copy for TpAttentionDims