pub struct K3HostShared {Show 14 fields
pub config: ModelConfig,
pub graph: K3Graph,
pub kda: KdaConfig,
pub mla: MlaConfig,
pub moe: LatentMoeConfig,
pub output_res_proj: DenseWeight,
pub output_res_norm: DenseWeight,
pub output_res_proj_meta: (WeightDtype, usize),
pub output_res_norm_meta: (WeightDtype, usize),
pub output_host: OnceLock<(Vec<f32>, Vec<f32>)>,
pub kda_kernels: OnceLock<K3KdaDecodeKernels>,
pub mla_kernels: OnceLock<K3MlaDecodeKernels>,
pub moe_kernels: OnceLock<K3MoeGemmKernels>,
pub attnres: Mutex<HashMap<DevicePtr, AttnResStream>>,
}Expand description
2026-09-25: One per loaded model; every decoder layer holds the same Arc.
Fields§
§config: ModelConfig§graph: K3Graph§kda: KdaConfig§mla: MlaConfig§moe: LatentMoeConfig§output_res_proj: DenseWeight§output_res_norm: DenseWeight§output_res_proj_meta: (WeightDtype, usize)§output_res_norm_meta: (WeightDtype, usize)§output_host: OnceLock<(Vec<f32>, Vec<f32>)>§kda_kernels: OnceLock<K3KdaDecodeKernels>2026-09-25: Resolved on the first CUDA KDA decode, then reused.
mla_kernels: OnceLock<K3MlaDecodeKernels>2026-09-25: Resolved on the first CUDA MLA decode, then reused.
moe_kernels: OnceLock<K3MoeGemmKernels>2026-09-25: Resolved at load for a checkpoint with packed experts, then reused.
attnres: Mutex<HashMap<DevicePtr, AttnResStream>>2026-09-25: One AttnRes stream per token in flight, carried across layers.
Keyed by the token’s residual pointer (hidden when that is null):
prefill runs every token through one layer before the next, and each
token has its own residual row, so each token keeps its own stream.
Auto Trait Implementations§
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more