Enum NgramTable 

pub enum NgramTable {
    Bf16(DenseWeight),
    Fp8(Fp8DenseWeight),
    Cached(Box<NgramRowCache>),
}

2026-09-25: One n-gram lookup table. Fp8 holds E4M3 rows with a per-row f32 scale, which the gather (batched_embed_fp8) applies.

Variants

Bf16(DenseWeight)

Fp8(Fp8DenseWeight)

Cached(Box<NgramRowCache>)

2026-09-25: NVMe-backed: a bounded set of rows is resident in a GPU-addressable arena. The host maps row ids to arena slots (NgramRowCache::resolve) and the same gather kernels read the arena by slot.

Implementations

impl NgramTable

pub fn quantize_bf16( w: &DenseWeight, rows: usize, dim: usize, gpu: &dyn GpuBackend, stream: u64, ) -> Result<Self>

2026-09-25: Quantize a BF16 table to FP8 on the GPU with quantize_bf16_to_fp8. w is not freed.