lmhead_overlay_routed

Function lmhead_overlay_routed 

Source
pub fn lmhead_overlay_routed(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    hidden: DevicePtr,
    seq_slot: DevicePtr,
    active: i32,
    rows_tab: DevicePtr,
    ids_tab: DevicePtr,
    n_tab: DevicePtr,
    logits: DevicePtr,
    m: u32,
    max_n_override: u32,
    h: u32,
    vocab: u32,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: In-place recompute of overridden logit columns in logits ([m, vocab], BF16 or FP32 to match kernel). One warp per (row, j), where j < max_n_override indexes the overridden ids of that row’s adapter slot; the kernel skips j >= n_tab[s] and ids >= vocab. hidden is [m, h] BF16; seq_slot is [m] i32, or null for the uniform active slot.