q12_batched_scratch_bytes

Function q12_batched_scratch_bytes 

Source
pub fn q12_batched_scratch_bytes(
    n: usize,
    chunk_len: usize,
    top_k: usize,
    mrope: bool,
) -> usize
Expand description

2026-09-25: Scratch bytes of the kernel-batched prefill staging for n streams of chunk_len tokens each. Both BufferSizes::from_config and check_kernel_batched_eligible call it (or the varlen form), so they agree on whether a batch fits. The terms follow prefill_b/batch_kernel.rs (MoE top-K area, n per-stream metadata blocks), prefill_b/stage_batched.rs (positions, three streams under MRoPE, slots, the block-table and seq_len pointer arrays, cu_seqlens, kv_lens) and the h_state_ptrs table.