Struct Predictor 

pub struct Predictor { /* private fields */ }

Implementations

impl Predictor

pub fn new( ctx: &CudaCtx, dims: PredictorDims, projection_seed: u64, ) -> Result<Self>

pub fn new_on_stream( stream: u64, dims: PredictorDims, projection_seed: u64, ) -> Result<Self>

pub fn project_q(&self, ctx: &CudaCtx, q: u64, q_proj: u64) -> Result<()>

2026-09-25: q_proj = q @ P. q is a device pointer to [num_q_heads, head_dim] BF16, q_proj to the [num_q_heads, r] BF16 output.

pub fn project_q_on_stream( &self, stream: u64, q: u64, q_proj: u64, ) -> Result<()>

2026-09-25: Self::project_q on an explicit stream.

pub fn project_kv_block( &self, ctx: &CudaCtx, layer: usize, block_id: usize, k_block: u64, ) -> Result<()>

2026-09-25: Write the low-rank keys of block block_id at layer into A_g. k_block is a device pointer to [block_size, num_kv_heads, head_dim] BF16. An out-of-range layer or block is an error.

pub fn project_kv_block_on_stream( &self, stream: u64, layer: usize, block_id: usize, k_block: u64, ) -> Result<()>

pub fn score_blocks( &self, ctx: &CudaCtx, q_proj: u64, k_lr_seq: u64, scores_out: u64, num_active_blocks: usize, ) -> Result<()>

2026-09-25: Score the first num_active_blocks blocks at k_lr_seq, a device pointer to [num_active_blocks, num_kv_heads, block_size, r] BF16. q_proj is [num_q_heads, r] BF16; scores_out receives [num_active_blocks] f32.

pub fn score_blocks_on_stream( &self, stream: u64, q_proj: u64, k_lr_seq: u64, scores_out: u64, num_active_blocks: usize, ) -> Result<()>

pub fn dims(&self) -> PredictorDims

pub fn a_g_dev_ptr(&self) -> u64