Module weights

Expand description

Weight loading from safetensors files (SBIO IORouter for filesystem I/O).

Modules§

adapter: Split a tensor name into (everything but its last numeric path segment, that segment as a number) so names sort NUMERICALLY on the index. embedders.2 must precede embedders.10; a plain lexicographic sort puts 10 first and silently mis-maps every table after the ninth. PEFT adapter loader: adapter_model.safetensors → WeightStore.
dequant_cpu: Public for kernel oracles: the CPU reference decoders every GPU dequant is held to. CPU reference dequantization for GGUF/GGML quantized weight blocks.
expert_stream: DeepSeek-V4.1 Flash expert streaming from the split GGUF, by positional read.

DeferredTensor: Where a skipped tensor lives, so a consumer can read it in place.
DerivedStore: Device buffers a loader derived from this store’s tensors.
GgufLoader: Loads weights from a GGUF file, dequantizing every tensor to BF16 on the GPU.
GgufShardSet: One resolved split-GGUF file set, in split.no order.
K3SafetensorsLoader
KvScaleCensus: 2026-09-28: Every attention layer whose scales resolve_kv_scale_keys finds, keyed by attention prefix.
KvScaleKeys: 2026-09-28: The two store keys one attention layer’s scales are read from.
SafetensorsLoader: Loads weights from safetensors files using mmap.
WeightStore: All model weights loaded onto the GPU, keyed by HuggingFace name.
WeightTensor: A weight tensor on the GPU.

KvScaleSpelling: 2026-09-28: How a checkpoint names one layer’s K/V scale pair.
WeightDtype: Data type of a weight tensor.

auto_detect_weight_prefix: Resolve the weight-key prefix a nested (multimodal) checkpoint uses.
config_from_gguf_dir: 2026-09-26: Build a ModelConfig from the .gguf that find_gguf picks in model_dir.
find_gguf: Locate the backbone GGUF weight file in dir. Returns the lexicographically-first non-mmproj *.gguf (also the first shard, *-00001-of-*, of a split file). The mmproj vision sidecar is excluded here and loaded separately (see sidecar::find_mmproj); a dir that is only an mmproj falls back to the first file so the caller still gets a path to error on.
find_gguf_shards: Resolve every shard of a split GGUF, given the first one.
is_ngram_table: Whether a tensor is an n-gram embedding TABLE — the huge blobs that are served off NVMe instead of being uploaded with the rest of the checkpoint.
kv_scale_census: 2026-09-28: Parse every name that ends in a known scale suffix to its attention prefix (most specific spelling first), then resolve each prefix with resolve_kv_scale_keys. Any layer that resolver rejects fails the census.
parse_expert_index: Parse expert index from tensor name (e.g. “model.layers.3.mlp.experts.42.gate_proj.weight” → 42).
resolve_kv_scale_keys: 2026-09-28: The scale keys of the attention layer at attn_prefix.

DeferHook: A model loader’s answer to “which tensors will I read from disk myself?”.