forward_full_attention

Function forward_full_attention 

Source
pub fn forward_full_attention<Q: QuantWeights>(
    gpu: &dyn GpuBackend,
    cfg: &Qwen35ForwardConfig,
    k: &Qwen35Kernels,
    layer: &FullAttentionLayer<'_, Q>,
    scratch: &FullAttentionScratch,
    kv: &LayerKvCache,
    inv_freq_ptr: DevicePtr,
    positions_ptr: DevicePtr,
    x_in: DevicePtr,
    cache_pos: u32,
    seq_len_attn: u32,
    stream: u64,
) -> Result<DevicePtr>
Expand description

2026-09-25: Runs one full-attention decoder layer for one token and returns scratch.x_out, which holds the layer’s output residual stream.