Expand description
2026-09-25: The expert-store format. pack_record and unpack_record build
and parse one record in memory, with no I/O. ExpertIndex is the store’s
manifest.json, from which a reader rebuilds the record spec and layout.
The layer files are written and read in expert_pack_fs.rs.
Owner: metrale-storage experts. Invariants:
ExpertIndex::loadandExpertFileReader::openreject a manifest whoseversionis notExpertRecordHeader::VERSION.
Structs§
- Expert
File Reader - 2026-09-25: Reads records back with positional reads (
tier::pio).expert_tier.rsandexpert_peer.rsopen stores through it. - Expert
File Writer - 2026-09-25: Creates one file per MoE layer, sized to
bytes_per_layer, writes records at their offsets, and writesmanifest.jsoninfinish. The files are opened without O_DIRECT. - Expert
Index - 2026-09-25: The store manifest,
manifest.jsonbeside the per-layer files: the geometry a reader needs to rebuild the spec and layout and open the files. - Proj
Data - 2026-09-25: One projection’s packed and scale bytes, as
pack_recordwrites them. - Proj
View - 2026-09-25: One projection’s sub-buffers inside a parsed record.
Functions§
- pack_
record - 2026-09-25: Build one
stride-byte record: the header at offset 0, each projection’s bytes at the spec’s offsets, zeros elsewhere. Fails whenstrideis belowspec.raw_bytes()or a projection’s lengths differ from the spec. - unpack_
record - 2026-09-25: Parse a record of at least
spec.raw_bytes()bytes into its header and projection views. Fails on a short buffer or a header whose magic or version differs.