requires_single_chunk_prefill

Function requires_single_chunk_prefill 

Source
pub fn requires_single_chunk_prefill(
    model_type: &str,
    kv_lora_rank: usize,
) -> bool
Expand description

2026-09-25: Must chunked prefill run as one chunk for this model?

True for every MLA model (kv_lora_rank > 0) except glm5_next and kimi_k3. The Qwen3 attention layer’s paged MLA prefill (layers/qwen3_attention/prefill/paged_mla.rs) attends over the K/V of the call’s own tokens only and refuses seq_len_start != 0, so a second chunk cannot run there.

glm5_next and kimi_k3 do not prefill through that layer: Glm5NextLayer::prefill (model-arch glm5next_layer/mod.rs) takes seq_len_start, and K3BoundLayer (model-arch kimi_k3/bound.rs) keeps the trait’s default prefill, which calls decode once per token. Both keep the scheduler’s chunk budget.