Expand description
2026-09-25: Multi-module MTP proposer: draft slot i runs its own
MtpHead (own weights and KV cache), as
modules[i].forward_one(previous draft token, previous module's hidden).
Owner: model-layers (MTP head). Invariants:
modulesis non-empty, and each state holds oneMtpProposerStateper module.- A propose runs
min(num_drafts, modules.len())modules, the first ones.
Structsยง
- Multi
Module MtpHead - 2026-09-25: Independent MTP modules, one per draft slot.
- Multi
Module MtpState - 2026-09-25: Per-sequence state for
MultiModuleMtpHead: oneMtpProposerStateper module, since the modules do not share a KV cache.