Expand description
2026-09-25: Host side of the tiled decode attention
(kernels/paged_decode_attn_tiled.cu, kernels/attention_finalize.cu in
this crate). A decode step is begin_step, which sets the FP32 running
state (m, l, o) to (-inf, 0, 0); one step_tile per tile of KV blocks; and
finalize, which writes o / l (0 where l is 0) as BF16.
Owner: storage (high-speed swap).
Invariants: the (m, l, o) buffers are allocated once in new for
dims.max_seqs sequences and reused by every step.