Module moe 

2026-09-25: The MoE feed-forward block MoeLayer: router, routed experts, optional shared expert, and the kernel handles and pointer tables its forward paths use.

Owner: model-layers (MoE). Invariants:

• Construction (new_with_hash) fails unless num_experts > 0, num_experts_per_tok <= num_experts, and both are within the sigmoid routing kernels’ fixed bounds.

Structs

MoeLayer

2026-09-25: The MoE feed-forward block. It is a component that layers call for their FFN step; it does not implement TransformerLayer.

Enums

GroupedRouting

2026-09-26: Which router arithmetic forward_fp8_grouped_decode_routed runs.

Constants

NVFP4_GROUPED_DECODE_MAX_ROWS

2026-09-27: Widest row count this path admits on the CUDA-core expert kernels, the grouped FP8 decode’s.

NVFP4_GROUPED_DECODE_TC_MAX_ROWS

2026-10-02: Widest row count on the tensor-core expert kernels, the grouped FP8 tensor-core decode’s: one verify of 128 sequences at one draft.

Functions

fp8_grouped_decode_shape_ok

2026-09-25: Shape admission for the scalar grouped kernels, without a GPU: fp8_grouped_decode_rows_ok(m, false) and fp8_grouped_decode_dims_ok.

moe_expert_fp8_act

2026-09-28: Whether the MoE experts decode with FP8 activations: false (W8A16) unless the serve published otherwise.

nvfp4_grouped_decode_shape_ok

2026-09-27: Shape admission without a GPU: m in 1..=max_rows (2026-10-02: the selected expert kernels’ widest, NVFP4_GROUPED_DECODE_MAX_ROWS or NVFP4_GROUPED_DECODE_TC_MAX_ROWS), hidden % 32 == 0 (gate+up reads 32-element chunks) and inter % 16 == 0 (down reads 16-element blocks), and a shared expert as wide as a routed one (the shared SiLU product shares the routed layout).

set_moe_expert_fp8_act

2026-09-28: Publish whether the MoE experts decode with FP8 activations (W8A8), as the weight-quantization policy decides for the expert modules. Returns the value in force; a caller that gets a different one should warn.