Module safetensors

Module safetensors 

Source
Expand description

2026-09-25: The one place that turns a safetensors data_offsets pair into a byte span.

Owner: metrale-core. Invariants:

  • A returned TensorSpan satisfies abs_offset + len <= file_len, computed without overflow; every other input is an error.

A checkpoint is third-party data, so every number in its header is untrusted until checked. The header declares each tensor as "data_offsets": [start, end] relative to the data section, and the naive end - start is a u64 subtraction that wraps on a crafted or truncated file: a reversed pair yields a length near u64::MAX.

Both header parsers call this: metrale_model_weights::fast_weights (fast_weights/header.rs, whose spans become pread windows) and metrale_storage::weight_peer (weight_peer/shard.rs, which builds the RDMA manifest).

Structs§

TensorSpan
2026-09-25: A validated tensor byte span: an absolute file offset and a length that is known to fit inside the file it came from.

Functions§

tensor_span
2026-09-25: Validate one tensor’s data_offsets against the file that declared it.