Expand description
2026-09-28: The declared precision plan: what a checkpoint’s quantization_config
declares, per linear layer, for the weight AND the input activations.
This is the one place that reads the precision half of quantization_config
(compressed-tensors config_groups, ModelOpt quantized_layers / quant_algo, and the
HF fp8 block scheme). Loaders and kernel dispatch ask DeclaredPrecisionPlan::resolve
for a module and run what it says by default. A lever may only go BELOW what the plan
declares; running above it is not a silent default either.
Owner: config (quantization). Invariants:
- Pure: no environment, no I/O. A malformed block is an error, never a guess.
- A module the plan does not cover (ignored, unmatched, or no quantization at all)
resolves to
LayerPrecision::UNQUANTIZED: 16-bit weights and activations.
Structs§
- Declared
Precision Plan - 2026-09-28: The per-layer precision a checkpoint declares.
- Layer
Precision - 2026-09-28: What one linear layer declares.
Nonemeans that operand is not quantized (16-bit). - Operand
- 2026-09-28: One quantized operand as declared.
- Rule
- 2026-09-28: One scheme and the modules it targets.
Enums§
- Granularity
- 2026-09-28: How many values share one scale.
- Ignore
Dialect - 2026-09-30: Which format’s matching rules an ignore / exclude entry follows.
- NumKind
- 2026-09-28: Element type of a quantized operand.
- Plan
Source - 2026-09-28: Which
quantization_configdialect the plan came from. - Scale
Timing - 2026-09-28: When an operand’s scales are computed.
- Target
- 2026-09-28: One target or ignore entry.