Expand description
2026-09-25: Expert identity and on-disk record geometry for MoE expert
streaming. A record holds one (moe_layer, expert)’s gate, up and down NVFP4
projections behind a fixed header; each MoE layer’s file holds
num_experts records at one stride, which the whole model shares.
Owner: metrale-storage experts. Invariants:
- Inside a record the header comes first, then gate, up and down (packed,
then scale, for each), every sub-buffer at a multiple of
sub_alignand none overlapping. record_strideisraw_bytesrounded up to a multiple offs_block_size.
Structs§
- Expert
Key - 2026-09-25: One expert’s record.
layeris the dense MoE-layer index (0..num_moe_layers), not the model layer;ExpertIndex::moe_layer_to_model_layermaps one to the other. - Expert
Layout - 2026-09-25: File geometry of an expert store: one file per MoE layer
(
ExpertIndex::file_name), each holdingnum_expertsrecords back to back atrecord_stride. - Expert
Record Header - 2026-09-25: The fixed, versioned header at the front of every expert
record. It carries what the dims do not give, each projection’s
weight_scale_2andinput_scale, plus layer, expert and shape so a mismatched file can be recognised. - Expert
Record Id - 2026-09-25: Dense record id across the model:
layer · num_experts + expert. - Expert
Record Spec - 2026-09-25: Where each sub-buffer of an expert record sits, relative to the
record base;
pack_recordandunpack_recordplace and read bytes by it. Every offset is a multiple ofsub_align: - Proj
Bytes - 2026-09-25: Byte sizes of one NVFP4 projection’s two buffers:
packed, two 4-bit values per byte (n·k/2), andscale, one byte pergroup_sizevalues (n·k/group_size). The per-projection scalarsweight_scale_2andinput_scaleare in the record header instead.
Enums§
- Proj
- 2026-09-25: The three projections of an expert. Their order is the order
of the sub-buffers in a record and of the header’s per-projection fields,
so changing it changes the on-disk format that
ExpertRecordHeader::VERSIONidentifies.