Struct AttnV41 

pub struct AttnV41 {
    pub cfg: AttnV41Cfg,
    /* private fields */
}

2026-09-25: The attention runtime: kernels, RoPE tables, and workspaces for up to max_tokens positions per call.

Fields

cfg: AttnV41Cfg

Implementations

impl AttnV41

pub fn decode_capturable(&self, w: &AttnV41LayerWeights) -> bool

2026-09-25: Whether this layer’s single-token step can be captured into a CUDA graph: false on kv and index sources, whose forward decides the compressor group and the index top-k on the host mid-forward (position parity, a score download). Every other layer is kernels and copies once its per-token inputs are read from the device.

pub fn decode_fixed_topk(&self, w: &AttnV41LayerWeights) -> usize

2026-09-25: The topk a captured step launches sparse_attn with: the widest selection of the layer’s class, so the launch argument is constant; decode_prep_role pads the selection with -1. A -1 entry adds an exact zero to its thread’s denominator sum and nothing to the output, so the result equals the eager step’s bit for bit.

pub fn invalidate_decode_uploads(&mut self)

2026-09-25: Forget what the captured decode step last uploaded; the next Self::decode_prep re-uploads. Call after any eager forward (it writes pos / head_pos / idx_dev for itself) and at a new step.

pub fn decode_prep( &mut self, w: &AttnV41LayerWeights, shared: &SharedV41, gpu: &dyn GpuBackend, start_pos: usize, stream: u64, ) -> Result<Option<DevicePtr>>

2026-09-25: The host half of a capturable layer’s single-token step at start_pos: the position into pos / head_pos and the padded selection (window slots, then the shared index selection, then -1) into the class’s selection buffer, each only when it differs from what is already there. Returns the compressed rows the captured sparse_attn reads (None on ratio-0 layers): a pointer the graph bakes, which the callers compare on every replay.

pub fn decode_prep_role( &mut self, ratio_positive: bool, shared: &SharedV41, gpu: &dyn GpuBackend, start_pos: usize, stream: u64, ) -> Result<Option<DevicePtr>>

2026-09-25: Self::decode_prep by selection class: ratio_positive layers read the window plus the shared index selection from idx_dev, the others the window alone from idx_dev_win; both classes can be prepared for one multi-layer capture. Fails for start_pos outside 1..max_seq.

pub fn decode_body( &self, gpu: &dyn GpuBackend, w: &AttnV41LayerWeights, st: &AttnV41LayerState, x: DevicePtr, rows_b: Option<DevicePtr>, stream: u64, ) -> Result<DevicePtr>

2026-09-25: The device half of a capturable layer’s single-token step: the projections, the window ring write at pos % window (slot read on the device), sparse attention over the padded selection and the output projection into the returned buffer. No host work and no per-token scalar arguments, so a captured graph replays for any position once decode_prep has updated the device inputs.

impl AttnV41

pub fn forward( &self, gpu: &dyn GpuBackend, w: &AttnV41LayerWeights, st: &mut AttnV41LayerState, shared: &mut SharedV41, x: DevicePtr, m: usize, start_pos: usize, stream: u64, ) -> Result<AttnV41Run>

2026-09-25: One layer’s attention for m tokens at start_pos: x is the normed input [m, dim] bf16; the output is [m, dim] bf16 in run.out. Fails for m outside 1..=max_tokens or a position past max_seq.

impl AttnV41

pub fn new(gpu: &dyn GpuBackend, cfg: AttnV41Cfg) -> Result<Self>

pub fn out_ptr(&self) -> DevicePtr

2026-09-25: The layer output buffer, bf16 [max_tokens, dim]: a pointer the captured decode steps bake.

pub fn free(self, gpu: &dyn GpuBackend) -> Result<()>