Struct NgramRowCache 

pub struct NgramRowCache {
    pub hits: u64,
    pub misses: u64,
    pub evictions: u64,
    /* private fields */
}

2026-09-25: One table’s backing files and its resident row cache.

Fields

hits: u64

misses: u64

evictions: u64

Implementations

impl NgramRowCache

pub fn open( path: &Path, scale_path: Option<&Path>, rows_total: u64, row_stride: usize, slots: usize, ) -> Result<Self>

2026-09-25: Open path as the backing store for a table of rows_total rows of row_stride bytes, caching slots of them in pinned GPU-addressable memory. scale_path supplies the per-row f32 scales of an FP8 table.

pub fn open_at( path: &Path, base_offset: u64, scale_path: Option<&Path>, rows_total: u64, row_stride: usize, slots: usize, ) -> Result<Self>

2026-09-25: As Self::open, with the table starting at base_offset in the file. Zero geometry or row_stride > BLOCK is an error.

pub fn open_segmented( shards: &[(PathBuf, u64)], rows_per_shard: u64, scale_path: Option<&Path>, row_stride: usize, slots: usize, ) -> Result<Self>

2026-09-25: As Self::open_at, for a table split into shards at arbitrary offsets, possibly in several files. shards[i] is shard i’s file and the offset of its first row; every shard holds rows_per_shard rows.

pub fn table_dev_va(&self) -> Result<u64>

2026-09-25: Device VA of the row arena, the table the gather kernels index by slot.

pub fn set_constant_scale(&mut self, scale: f32) -> Result<()>

2026-09-25: Give every slot the same scale, for an FP8 table with one scale for all rows. The scale arena is filled here and never faulted.

Errors

If the scale arena cannot be allocated.

pub fn scale_dev_va(&self) -> Result<Option<u64>>

2026-09-25: Device VA of the [slots] f32 scale array; None for a BF16 table.

pub fn stats(&self) -> (u64, u64, u64)

pub fn resolve( &mut self, row_ids: &[u64], out_slots: &mut Vec<u32>, ) -> Result<()>

2026-09-25: Resolve row_ids to slot indices, faulting misses in from NVMe.

Every returned slot stays pinned until Self::end_batch, so a later resolve in the same batch cannot evict a row the gather has yet to read. An id at or past the table’s row count, every slot pinned, or a failed read is an error.

pub fn end_batch(&mut self)

2026-09-25: Release the batch’s pins; call after the gather kernels are issued.