Module decode_meta

Module decode_meta 

Source
Expand description

2026-09-25: Fixed-stride batched-decode metadata layout, derived from the serve max_batch_size.

sizes.rs sizes scratch from it, and metrale-model-engine’s upload_batch_metadata_fixed / _at take their offsets from it. Byte offsets, with R = rows: positions u32 [0, 4R) seq_slot i32 [4R, 8R) (per-request LoRA slot) slots i64 [8R, 16R) seq_lens i32 [16R, 20R) (unused [20R, 24R)) block_tbl i32 [24R, 24R + R·max_blocks·4)

Owner: gpu-runtime. Invariants:

  • rows() >= DECODE_META_MIN_ROWS.
  • The regions do not overlap, and slots_off() is a multiple of 8.

Structs§

DecodeMetaLayout
2026-09-25: The decode-metadata layout for rows rows (see the module docs).

Constants§

DECODE_META_MAX_ROWS
2026-09-25: The largest serve max_batch_size accepted; serve_load.rs refuses a larger one at startup. The layout itself has no upper bound.
DECODE_META_MIN_ROWS
2026-09-25: The row floor: rows = max(DECODE_META_MIN_ROWS, max_batch_size).