Struct WeightQuantPolicy

Source

pub struct WeightQuantPolicy<'a> { /* private fields */ }

Expand description

2026-09-28: The per-layer answers for one model: its declared plan under the published tier.

Implementations§

Source §

impl<'a> WeightQuantPolicy<'a>

Source

pub fn new( tier: WeightQuantTier, plan: &'a DeclaredPrecisionPlan, caps: KernelCaps, ) -> Self

2026-09-28: The policy for a model whose quantization_config parsed to plan.

Source

pub fn for_checkpoint( tier: WeightQuantTier, qc: Option<&'a QuantizationConfig>, caps: KernelCaps, ) -> Self

2026-09-28: The policy for a checkpoint whose parsed quantization_config is qc; None (no block) declares nothing.

Source

pub fn tier(&self) -> WeightQuantTier

2026-09-28: The tier in force.

Source

pub fn follows_plan(&self) -> bool

2026-09-28: Whether answers follow the checkpoint: the declared tier on a checkpoint that declares something.

Source

pub fn declared(&self, module: &str) -> LayerPrecision

2026-09-28: What module declares.

Source

pub fn nvfp4_act(&self, module: &str) -> Nvfp4Act

2026-09-28: The stamp for module’s NVFP4 weight (QuantizedWeight::act): A4 where the checkpoint declares FP4 activations, Wide where it declares any other (so no FP4 activations run below it), unstamped under nvfp4 or with no declaration.

Source

pub fn wants_fp8_weights(&self, module: &str) -> bool

2026-09-28: Whether the policy asks for module’s FP8 weights to be served as FP8, not requantized to NVFP4 at load: under declared, every weight the checkpoint declares FP8. A load site without an FP8 decode arm for the declared scale layout still requantizes, logs it, and stamps the NVFP4 copy Wide.

Source

pub fn fp8_decode_act(&self, module: &str) -> Option<ActFormat>

2026-09-28: The decode activations of module when it is served at its FP8 weights (None when the policy does not ask for them): FP8 where the checkpoint declares FP8 activations and the W8A8 decode kernels are present, BF16 otherwise (above declared, until those kernels land). This is WHEN; the W8A8 family says whether it CAN serve a given weight and row count.

Source

pub fn fp8_block_scaled_decode_act(&self, module: &str) -> Option<ActFormat>

2026-09-28: Self::fp8_decode_act for a non-expert module whose FP8 weight is 128x128 block-scaled: FP8 only when KernelCaps::w8a8_block_scaled_decode is set as well, BF16 (W8A16) otherwise. An expert module answers as fp8_decode_act does.

Source

pub fn declares_fp8_activations(&self, module: &str) -> bool

2026-09-28: Whether the policy serves module at its FP8 weights and the checkpoint declares FP8 activations for it, whatever the kernels. Where the W8A8 answer is BF16, the layer runs above its declared activations until its W8A8 path is validated, which the load log states.

Source

pub fn lifts_block_scaled_w8a8( &self, experts: &[String], experts_decode_fp8: bool, ) -> bool

2026-10-03: Whether the block-scaled attention/GDN W8A8 may run although KernelCaps::w8a8_block_scaled_decode is off. That cap is held because, stacked with the expert W8A8 on Qwen/Qwen3.6-35B-A3B-FP8, it flipped a greedy tie. Only a MoE checkpoint that declares FP8 activations on none of its experts (the NVFP4 35B, whose experts are NVFP4) has no such stack. A dense checkpoint, or the FP8 35B with its experts held at BF16 activations, keeps W8A16, as the combination is unvalidated there.

Source

pub fn lm_head(&self) -> LmHeadChoice

2026-09-28: The head --lm-head-dtype default takes: under declared, the declared format, except a declared FP8 head while KernelCaps::fp8_lm_head_batched is false; the engine’s per-model default otherwise, or for a format with no head kernel.