read_k_lr_slot

Function read_k_lr_slot 

Source
pub fn read_k_lr_slot(
    ctx: &CudaCtx,
    pred: &Predictor,
    layer: usize,
    block: usize,
) -> Result<Vec<bf16>>
Expand description

2026-09-25: Copy the A_g slot of (layer, block) to the host, as BF16 [num_kv_heads, block_size, r], and synchronise the stream.