apply_lora_delta

Function apply_lora_delta 

Source
pub fn apply_lora_delta(
    gpu: &dyn GpuBackend,
    kernels: &LoraKernels,
    pair: &LoraPair,
    x: DevicePtr,
    base_out: DevicePtr,
    m: u32,
    lora_xa: DevicePtr,
    lora_delta: DevicePtr,
    stream: u64,
) -> Result<()>
Expand description

2026-09-25: base_out[m, n_out] += scale * (x[m, k_in] @ a^T) @ b^T for one LoraPair.

x rows must be contiguous with stride k_in * 2 bytes and base_out rows with stride n_out * 2 bytes; a strided caller loops with m = 1 on offset pointers. lora_xa must hold m * max_rank BF16 and lora_delta m * n_out BF16.

Paths: m == 1 runs two GEMVs; 1 < m <= lora_gemv_max_m() runs one single-row delta per row; larger m runs the tensor-core GEMMs, with the fused expand-and-fold when both gemm_tc handles resolved, and dense_gemm when gemm_tc is missing. Every unfused path ends with bf16_scaled_add.