Struct NgramEmbedding 

pub struct NgramEmbedding {
    pub dims: NgramDims,
    pub word: DenseWeight,
    pub tables: Vec<NgramTable>,
    pub projs: Vec<DenseWeight>,
    /* private fields */
}

2026-09-25: GPU n-gram embedding from batched_embed gathers, dense_gemm_bf16_pipelined projections and bf16_scaled_add accumulation: per call 1 + T gathers, T GEMMs and T + 1 scaled adds, T = num_tables().

Fields

dims: NgramDims

word: DenseWeight

2026-09-25: Base word embedding, [vocab, hidden] BF16.

tables: Vec<NgramTable>

2026-09-25: The K*(N-1) lookup tables in index order (ngram-2)*K + split, each [table_rows(index), table_dim].

projs: Vec<DenseWeight>

2026-09-25: Per-table projections, [hidden, table_dim] BF16.

Implementations

impl NgramEmbedding

pub fn new( dims: NgramDims, word: DenseWeight, tables: Vec<NgramTable>, projs: Vec<DenseWeight>, max_tokens: usize, gpu: &dyn GpuBackend, ) -> Result<Self>

pub fn embed( &mut self, ctx_tokens: &[u32], seq_len: usize, out: DevicePtr, gpu: &dyn GpuBackend, stream: u64, ) -> Result<()>

2026-09-25: Embedding of the last seq_len tokens of ctx_tokens (earlier context tokens, then the new ones), written to out as [seq_len, hidden] BF16. Errors when seq_len exceeds the staging or ctx_tokens, or an id does not fit u32.