build_overlay

Function build_overlay 

Source
pub fn build_overlay(
    gpu: &dyn GpuBackend,
    kernels: &OverlayKernels,
    slot: &OverlayRawSlot,
    served_embed: DevicePtr,
    served_lmhead: DevicePtr,
    vocab: usize,
    h: usize,
    tied: bool,
    stream: u64,
) -> Result<Option<EmbedOverlay>>
Expand description

2026-09-25: Build one slot’s EmbedOverlay: clamp the trainable ids, diff and compact the embed rows, build the slot_map, and, for an untied head (!tied) with its own base, the lm_head rows. The staged buffers are freed when an overlay is returned. Ok(None) when the slot has no embed base or overrides no row. An error when the overlay kernels are not loaded. Model-engine passes tied when the lm_head shares the embed buffer or is served quantized; TokenOverlaySet::from_slots then points the lm_head at the embed rows.