pub struct LoraKernels {
pub gemv_k: KernelHandle,
pub gemm_tc_k: KernelHandle,
pub gemm_tc_acc_k: KernelHandle,
pub gemm_k: KernelHandle,
pub scaled_add_k: KernelHandle,
pub bgmv_shrink_k: KernelHandle,
pub bgmv_expand_fold_k: KernelHandle,
pub moe_down_shrink_k: KernelHandle,
pub moe_down_expand_fold_k: KernelHandle,
pub moe_gather_shrink_k: KernelHandle,
pub moe_gather_expand_fold_k: KernelHandle,
}Expand description
2026-09-25: The kernel handles the LoRA launchers use, resolved when an adapter is installed.
Module names follow the [modules] table of kernels/gb10/common/KERNEL.toml
(dense_gemv_bf16.cu is gemv, dense_gemm_tc.cu is gemm_tc, dense_gemm_bf16.cu is
gemm); residual_add, lora_bgmv, moe_lora_grouped_down and moe_lora_gather_bgmv are
file stems.
Fields§
§gemv_k: KernelHandle§gemm_tc_k: KernelHandle§gemm_tc_acc_k: KernelHandle2026-09-25: Fused expand and fold (C += scale * A @ B^T). KernelHandle(0) when missing;
the unfused gemm_tc + scaled_add pair is used instead.
gemm_k: KernelHandle§scaled_add_k: KernelHandle§bgmv_shrink_k: KernelHandle2026-09-25: The per-request routed kernels of apply_lora_bgmv: shrink, then expand and
fold.
bgmv_expand_fold_k: KernelHandle§moe_down_shrink_k: KernelHandle2026-09-25: MoE expert LoRA over expert-sorted rows, keyed on expert through the device
expert_offsets: shrink, then expand and fold. KernelHandle(0) when missing; the launcher
in moe_lora_grouped.rs then returns an error rather than launching.
moe_down_expand_fold_k: KernelHandle§moe_gather_shrink_k: KernelHandle2026-09-25: MoE expert LoRA over unsorted, slot-major rows, keyed on expert through the
per-slot indices array: shrink, then expand and fold. KernelHandle(0) when missing; the
launcher in moe_lora_grouped.rs then returns an error rather than launching.
moe_gather_expand_fold_k: KernelHandleImplementations§
Source§impl LoraKernels
impl LoraKernels
pub fn new(gpu: &dyn GpuBackend) -> Result<Self>
Trait Implementations§
Source§impl Clone for LoraKernels
impl Clone for LoraKernels
Source§fn clone(&self) -> LoraKernels
fn clone(&self) -> LoraKernels
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more