Module weight_quantization 

2026-09-28: --weight-quantization: the precision each linear layer runs at, and the one policy that loaders and kernel dispatch ask for it.

• declared (the default) follows the checkpoint’s quantization_config (DeclaredPrecisionPlan) per layer, for the weights and for the input activations.
• nvfp4 is the engine as it ran before the plan existed. FP8-declared projections are requantized to NVFP4 at load. Decode runs 16-bit activations, and FP4 activations only under --w4a4-downcast (W4a4Downcast). The dense-FFN MMQ prefill runs FP4 activations on every NVFP4 FFN.

The serve publishes a WeightQuantTier once. Each loader builds a WeightQuantPolicy from it and the model’s plan, and stamps what the policy answers on the weights it builds (Nvfp4Act). Kernel dispatch reads those stamps and the published tier, and nothing else.

Owner: config (quantization). Invariants:

• Pure: no environment, no I/O, no process state.
• Under nvfp4 no answer depends on the plan.
• A checkpoint that declares nothing (DeclaredPrecisionPlan::is_undeclared) gets the nvfp4 answers under both tiers. Load-time quantization of a 16-bit checkpoint is governed by its own flags, not this one.

Structs

KernelCaps

2026-09-28: Which kernels the policy may rely on. The W8A8 decode family reports its presence here (metrale_model_layers::layers::kernel_caps).

WeightQuantPolicy

2026-09-28: The per-layer answers for one model: its declared plan under the published tier.

WeightQuantTier

2026-09-28: What the serve publishes: the tier and, under nvfp4, its W4A4 lever.

Enums

ActFormat

2026-09-28: The activation format a layer runs at in decode.

LmHeadChoice

2026-09-28: What --lm-head-dtype default resolves to (WeightQuantPolicy::lm_head).

LmHeadFormat

2026-09-28: The lm_head format --lm-head-dtype default resolves to under declared.

Nvfp4Act

2026-09-28: The activation format the policy set for one NVFP4 weight, which the loader stamps on it (QuantizedWeight::act).

W4a4Downcast

2026-09-28: --w4a4-downcast and --w4a4-downcast-wide, which exist only under the nvfp4 tier. They run every NVFP4 small-M projection’s activations as NVFP4 (W4A4), whatever the checkpoint declares.

WeightQuantization

2026-09-28: The --weight-quantization tiers.

Functions

is_expert_module

2026-09-28: Whether module is a MoE expert projection (routed or shared), which the KernelCaps::w8a8_moe_decode kernels serve.