Expand description
2026-09-30: How a weight-reading node executes its DECLARED formats on a device. The declared precision is never changed here; what varies per device is the instruction that carries it:
- the activation format picks the MMA kind (BF16 activations: BF16 MMA, any weight format widened in registers; FP8: the FP8 MMA; NVFP4: the block-scaled FP4 MMA);
- a device without the FP4 MMA but with the FP8 one runs NVFP4 operands through the exact
E2M1 -> E4M3 conversion (
(q & 0x80808080) | ((q & 0x70707070) >> 2)is value x 2^-6 for all 16 codes; the 2^6 folds into the group scale), block scales applied in FP32 per K=16 step: the same products as the declared W4A4, not an upcast; - anything else has no path, which the plan states.
A device’s instruction is not enough: a node runs on the FP4 MMA only where the device’s class
compiles an FP4 block-scale kernel for it (super::avail::without_fp4_kernel); elsewhere it
takes the same path as on a device without that MMA. node_exec is the one answer per node:
the report’s execution columns, its “FP4 costing” row and every node’s cost read it.
Owner: metrale-circuit (hardware).
Invariants: no silent upcast. An activation format with no native or exact path is
Exec::NoPath, never re-planned at a wider format.
Enums§
- Exec
- 2026-09-30: The execution of one node’s declared formats.
Functions§
- exec_of
- 2026-09-30: How
deviceexecutes a node ofweightreadingactivation. NVFP4 weights under FP8 activations (W4A8 with group-16 E4M3 scales) have no single-instruction MMA on any device unlessnative_mmasays so (fp4_fp8_nvfp4_scaled); they take the exact E4M3 path. - fp4_
fallback - 2026-10-01: How
deviceruns an NVFP4 activation without an FP4 block-scale kernel: on the FP8 MMA through the exact conversion where it has one, else no path;Exec::NoFp4Kernelwhen the device has the FP4 MMA and only the kernel is missing. - kind_of
- 2026-09-30: The MMA kind the activation format needs.
- node_
exec - 2026-10-01: How node
nofcruns ondevice(weight-reading nodes only): its declared formats on the device’s instructions (exec_of), orfp4_fallbackwhen the class compiles no FP4 block-scale kernel for it (no_fp4_kernel). - pair_
name - 2026-09-30: The declared pair as
W4A16-style text.