Module nllb 

2026-09-25: Served NLLB-200 / M2M-100 encoder-decoder translation model.

NllbGpuModel implements crate::traits::Model, so an NLLB checkpoint is served through the same scheduler and API as the decoder-only models, with its weights from the standard --model store. The factory builds it for model_type m2m_100 or nllb.

The model owns all its KV and does not use the scheduler’s paged block cache: per sequence, a decoder self-attention cache that grows one row per token and a cross-attention cache computed once from the encoder (see kv). Logits are bf16.

Owner: model-engine. Invariants: none beyond the types.

Structs

NllbGpuModel

2026-09-25: The served NLLB encoder-decoder model. Fields are immutable after construction, behind a Mutex, or atomic. The decode scratch (dec) and the LoRA gate (lora_active) are shared by every sequence, so callers must not forward two sequences at once.

NllbLang

2026-09-25: Resolved translation tokens for the deployment. src_lang_id and tgt_lang_id are the defaults from --src-lang/--tgt-lang; a request’s own non-zero language ids override them. The factory sets decoder_start_id and eos_id to the config’s eos_token_id and pad_id to 1.