launch_k3_mla_decode_token

Function launch_k3_mla_decode_token 

Source
pub fn launch_k3_mla_decode_token(
    gpu: &dyn GpuBackend,
    kernels: &K3MlaDecodeKernels,
    q: &mut [f32],
    k: &mut [f32],
    v: &[f32],
    g: &[f32],
    kv: &mut MlaKv,
    cfg: &MlaConfig,
    pos: usize,
    theta: f32,
    stream: u64,
) -> Result<Vec<f32>>
Expand description

2026-09-25: One decode token against a host MlaKv: writes the roped q/k back, appends to kv, and uploads the whole KV history.