Struct AttnMetadataDev 

pub struct AttnMetadataDev {
    pub positions: DevicePtr,
    pub positions_h: DevicePtr,
    pub positions_w: DevicePtr,
    pub slot: DevicePtr,
    pub seq_len: DevicePtr,
    pub block_table: DevicePtr,
    pub max_blocks_per_seq: u32,
    pub num_seqs: u32,
    pub seq_slot: DevicePtr,
    pub moe_row_adapter: DevicePtr,
}

2026-09-25: Device addresses of one pass’s attention metadata, uploaded by the model before its layer loop. On a decode pass each row is one sequence, with one entry (or one block-table row) per row in each array. A prefill pass has num_seqs == 1 and one positions and slot entry per token.

Fields

positions: DevicePtr

2026-09-25: Positions, u32, one per token row. Under MRoPE this is the temporal (T) stream.

positions_h: DevicePtr

2026-09-25: MRoPE height (H) positions. The same pointer as positions except on a prefill pass that uses MRoPE (prefill_b/forward_layers.rs).

positions_w: DevicePtr

2026-09-25: MRoPE width (W) positions, set like positions_h.

slot: DevicePtr

2026-09-25: KV write slot of each token row, i64.

seq_len: DevicePtr

2026-09-25: Sequence lengths, one 32-bit entry per row. On decode each is the length including the token being decoded (seq_len + 1).

block_table: DevicePtr

2026-09-25: Block tables, 32-bit entries, row-major.

max_blocks_per_seq: u32

2026-09-25: Entries per row of block_table.

num_seqs: u32

2026-09-25: Rows in the batch, padding rows included; 1 for single-sequence decode.

seq_slot: DevicePtr

2026-09-25: Per-row LoRA adapter slot, i32 (one per token on prefill), at a fixed address, so the batched bgmv can run inside a captured decode graph. A negative slot applies no delta (lora_bgmv.cu); padding rows are -1. DevicePtr(0) when no LoRA weights are loaded, or on a single-request pass whose request uses the active adapter (upload_seq_slot_uniform); the apply sites then skip the bgmv.

moe_row_adapter: DevicePtr

2026-09-25: Per-row adapter map for the batched-decode MoE expert fold, [num_seqs] i32 in the fixed-address TransformerModel::moe_row_adapter_buf, built by crate::lora::build_moe_row_adapter_decode. < 0 means no fold on that row; >= 0 folds the active adapter’s per-expert delta. DevicePtr(0) when no LoRA weights are loaded and on every path that does not upload it. Unlike seq_slot, no row resolves a negative slot to the active adapter.

Implementations

impl AttnMetadataDev

pub fn rows_from(self, row: usize) -> Self

2026-10-01: The metadata of rows row.. of a prefill pass: every per-row array (positions, slots, the LoRA slot) starts row rows later; the block table and sequence length are the pass’s own.

Trait Implementations

impl Clone for AttnMetadataDev

fn clone(&self) -> AttnMetadataDev

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for AttnMetadataDev