Struct PleLayer 

pub struct PleLayer { /* private fields */ }

Implementations

impl PleLayer

pub fn snapshot_aux( &self, st: &PleSeqState, gpu: &dyn GpuBackend, stream: u64, ) -> Result<Vec<u8>>

2026-09-25: The aux blob: hist_len (u32), the history (u32 each), then the conv state’s f32 bytes, all little-endian.

pub fn restore_aux( &self, st: &mut PleSeqState, blob: &[u8], gpu: &dyn GpuBackend, stream: u64, ) -> Result<()>

2026-09-25: Restore a blob from Self::snapshot_aux and clear prestaged_va. Errors when the blob’s size does not match.

impl PleLayer

pub fn prestage( &self, st: &mut PleSeqState, tokens: &[u32], gpu: &dyn GpuBackend, stream: u64, ) -> Result<()>

2026-09-25: The per-step host work of a decode step: the n-gram hash, the row gather (gather_host, which uploads the slots into slots_dev from host memory) and the history advance. It runs through decode_prestage before any graph replay or capture; the next forward consumes prestaged_va and does not advance the history again.

pub fn release_seq_state( &self, st: &mut PleSeqState, gpu: &dyn GpuBackend, ) -> Result<()>

2026-09-25: Free one sequence’s PLE carry; conv is a bare DevicePtr, so dropping PleSeqState frees nothing. Idempotent: conv is nulled, and the state cleared, even when the free fails, whose error is returned.

impl PleLayer

pub fn new( dims: PleIdDims, head_dim: usize, hidden: usize, hc_mult: usize, k_size: usize, dilation: usize, eps: f32, weights: PleWeights, table: NgramTable, max_tokens: usize, gpu: &dyn GpuBackend, ) -> Result<Self>

pub fn new_seq_state(&self, gpu: &dyn GpuBackend) -> Result<PleSeqState>

2026-09-25: Allocate one sequence’s PLE carry (conv buffer + empty history). The conv buffer is not initialised here; the first forward or prestage sees the empty history and runs reset.

pub fn rearm(&self, st: &mut PleSeqState)

2026-09-25: Restore the prestaged state after a failed CUDA-graph capture attempt (the eager re-run calls forward again, and the first call consumed prestaged_va).

pub fn forward_row( &self, st: &mut PleSeqState, highway_row: DevicePtr, ids: &[u32], ctx: &ForwardContext<'_>, stream: u64, ) -> Result<()>

2026-09-25: One highway row with an explicit id: the multi-seq decode entry (ctx.host_token_ids holds the whole batch; the caller slices). It never consumes a prestage.

pub fn forward( &self, st: &mut PleSeqState, highway: DevicePtr, num_tokens: usize, fresh: bool, ctx: &ForwardContext<'_>, stream: u64, ) -> Result<()>

2026-09-25: Inject into highway [T, hc_mult*hidden] FP32, in place. fresh starts a new sequence (prefill from position 0).