Struct MoeWeights 

pub struct MoeWeights {
    pub gate: DenseWeight,
    pub shared_expert: ExpertWeight,
    pub shared_expert_gate: DenseWeight,
    pub experts: Vec<ExpertWeight>,
    pub router_pre_norm: Option<DenseWeight>,
    pub correction_bias: Option<DenseWeight>,
}

2026-09-25: Weights of one MoE block.

Fields

gate: DenseWeight

2026-09-25: Router gate, BF16, one row of hidden_size per router logit.

shared_expert: ExpertWeight

shared_expert_gate: DenseWeight

2026-09-25: Shared-expert gate [1, hidden_size] BF16: the shared expert’s output is scaled by sigmoid(dot(input, gate)), or by 1 when this pointer is NULL.

experts: Vec<ExpertWeight>

2026-09-25: One entry per expert id; experts this rank does not hold are ExpertWeight::null().

router_pre_norm: Option<DenseWeight>

2026-09-25: RMSNorm weight applied to the router input before the gate GEMV (MoeLayer::router_input). The Gemma-4 loader sets it to the BF16 vector router.scale * hidden_size^-0.5; None feeds the gate the input unchanged.

correction_bias: Option<DenseWeight>

2026-09-25: Per-expert F32 bias [num_experts] added to the router scores for top-k selection.