Module latent_moe 

2026-09-25: Re-exports the Kimi K3 LatentMoE CPU reference (latent down projection, sigmoid top-k router, SiTU-GLU experts, up projection), metrale_model_weights::kimi_k3_host::latent_moe.

Owner: model-arch, Kimi K3. Invariants: none beyond the types.

Structs

LatentMoeConfig

Functions

expert_situ

One SiTU-GLU expert: down( situ(w1 x, w3 x) ). Weights are [out, in].

latent_moe_forward

Routed latent path + optional shared expert (identity-scale for tests).

mix_routed_experts

Mix selected SiTU-GLU experts into latent. Empty w1 means packed-only (no host dequant) — CUDA grouped GEMM must have run instead.

moe_from

sigmoid_topk

noaux_tc: sigmoid(logits) for mix weights; scores + bias only ranks.