pub fn apply_lora_bgmv(
gpu: &dyn GpuBackend,
kernels: &LoraKernels,
route: &LoraRoute,
x: DevicePtr,
base_out: DevicePtr,
seq_slot: DevicePtr,
n: u32,
x_row_stride: u32,
out_row_stride: u32,
lora_xa: DevicePtr,
stream: u64,
) -> Result<()>Expand description
2026-09-25: Per-request routed LoRA delta over n rows, each naming its adapter slot in
seq_slot[n] (i32; a negative slot gets no delta):
out[i, :] += scale_s * (x[i, :] @ A_s^T) @ B_s^T with s = seq_slot[i].
Two launches (kernels/gb10/common/lora_bgmv.cu): the shrink writes BF16 xa, and the expand
rounds each delta to BF16 before adding scale_table[s] * delta in FP32, the order of
bf16_scaled_add. A slot whose table entry is 0 does not adapt this module. Both stages run
at route.max_rank.
Strides are in elements: x_row_stride between x rows (at least k_in) and
out_row_stride between base_out rows (at least n_out), so the fold can land inside an
interleaved buffer. lora_xa must hold n * max_rank BF16. The launch arguments follow the
kernels’ parameter order; the launch does not check their types.