Module tp 

2026-09-25: GLM-5.3-Flash KDA tensor-parallel shard plan: head-parallel, one all-reduce after o_proj.

Owner: model-arch (GLM-5.3-Flash KDA). Invariants:

• KdaTpPlan::new refuses tp_rank >= tp_size, a head count not divisible by tp_size, and a local q|k width that is not a multiple of 256.
• Sharded tensors split into equal, contiguous per-rank ranges ordered by rank.

The plan is pure data, so the per-rank row arithmetic is tested without a GPU; tp_bind applies it. Each rank owns a contiguous head range, o_proj is row-parallel, and one all-reduce follows it, as in the GDN head-parallel helpers (crate::tp_shard::gdn). Unlike GDN, KDA’s q/k/v_proj and q/k/v_conv1d are separate tensors on disk, so each is sliced on its own and no segmented copy is needed.

• A_log is per head and dt_bias per channel, so they shard in different units.
• o_norm is [head_dim], shared by every head, so it is replicated.
• f_a/g_a are down-projections [rank, hidden] and are replicated; only the _b up-projections carry head structure.
• o_proj is [hidden, heads * head_dim] sliced on its input dimension; each rank’s output is partial until the all-reduce.

Structs

KdaTensorPlan

2026-09-25: One tensor’s placement, in [rows, row_elems] terms. For KdaShard::ChannelCols the sharded axis is row_elems.

KdaTpPlan

2026-09-25: The per-rank shard plan for one KDA block.

Enums

KdaShard

2026-09-25: How one KDA tensor maps onto TP ranks.