pub fn requires_single_chunk_prefill(
model_type: &str,
kv_lora_rank: usize,
) -> boolExpand description
2026-09-25: Must chunked prefill run as one chunk for this model?
True for every MLA model (kv_lora_rank > 0) except glm5_next and
kimi_k3. The Qwen3 attention layer’s paged MLA prefill
(layers/qwen3_attention/prefill/paged_mla.rs) attends over the K/V of the
call’s own tokens only and refuses seq_len_start != 0, so a second chunk
cannot run there.
glm5_next and kimi_k3 do not prefill through that layer:
Glm5NextLayer::prefill (model-arch glm5next_layer/mod.rs) takes
seq_len_start, and K3BoundLayer (model-arch kimi_k3/bound.rs) keeps the
trait’s default prefill, which calls decode once per token. Both keep the
scheduler’s chunk budget.