Module attention_ref 

2026-09-25: Host reference for the tiled online-softmax decode attention of kernels/paged_decode_attn_tiled.cu and kernels/attention_finalize.cu.

Owner: metrale-storage. Invariants: none beyond the types.

Structs

AttnState

2026-09-25: Running state. m and l are [num_seqs, num_q_heads], o is [num_seqs, num_q_heads, head_dim]; new sets m = -inf and l = o = 0.

Functions

finalize_ref

step_tile_ref

2026-09-25: One tile update over the paged layout [num_blocks, block_size, num_kv_heads, head_dim], with no causal mask. tile_blocks is [num_seqs, tile_capacity], tile_block_counts is [num_seqs]. It runs the kernel’s per-token recurrence in fp32, but sums each dot product sequentially and uses exp where the kernel uses a warp-tree sum and __expf.