Module moe_v41 

2026-09-25: DeepSeek-V4.1 Flash MoE on streamed, still-quantized experts.

The routed experts stay in their GGUF blocks: gate and up run on Q2_K kernels, down on Q3_K kernels. Per token the router picks topk of n_routed experts, the ExpertLru gathers those experts’ raw slices into its device-visible slots (misses are read from the ExpertSource), and the kernels read the blocks directly with the activation quantised to q8_1: GEMVs for groups of up to eight rows, and by default MMQ above. The shared expert is resident, bf16 or K-quant (ResidentMat).

Routing matches deepseek_v41_ref::moe::gate: f32-accumulated router logits, scores sqrt(softplus(logit / gate_temp)), the top-k chosen by score + gate_bias with a stable sort, and the weights the unbiased scores, renormalised when norm_topk_prob and scaled by route_scale. By default the selection runs on the CPU from the downloaded logits; device_route holds the device version.

Tested in moe_v41_tests.rs on synthetic Q2_K / Q3_K experts against the reference gate and a CPU emulation of the numerics.

Owner: model-arch (DeepSeek-V4.1 MoE). Invariants: none beyond the types.

Structs

MoeV41

MoeV41Cfg

MoeV41LayerWeights

2026-09-25: One layer’s resident MoE weights: the router and the shared expert.

MoeV41Stage

2026-09-25: What stage_m1 leaves for compute_m1: the distinct expert count and the routing (for callers and diagnostics), with the host-span timing.

MoeV41Timing

2026-09-25: Where one call’s time went (wall clock, host side).

RouterWeights

2026-09-25: The router of one layer alone: what predicting a layer’s selection from an earlier layer’s input needs (see MoeV41::predict_launch).

Functions

bf16_bytes

2026-09-25: v rounded to bf16 (nearest, ties to even) as little-endian bytes, for callers staging inputs.

prefetch_k

2026-09-25: METRALE_DS41_PREFETCH_K, read once (default 0, off): how many of the next layer’s predicted experts to start reading while this layer computes. It acts only at one token, with the host selection, and with the reader pool (METRALE_DS41_READER_POOL=1).

route_from_logits

2026-09-25: deepseek_v41_ref::moe::gate on f32 logits: returns (weights[tokens, topk], indices[tokens, topk]), each token’s picks in descending score + bias order, ties to the lower expert id. Panics on a NaN key.

shared_side

2026-09-25: METRALE_DS41_SHARED_SIDE, read once (default on; 0 turns it off): at one token the shared expert runs on a side stream while the main stream runs the router and the host picks the experts. Its output is added into acc after the routed experts, the same point as on the main stream, so the accumulation order does not change.