Module engine 

2026-09-25: A greedy generate loop for one request over the Model trait: prefill, then decode, taking each token with argmax_on_device.

Owner: metrale-model-engine. Invariants:

• The engine holds no state: generate and generate_streaming allocate a fresh sequence per call and call free_sequence on it whether generation returned Ok or Err.

Structs

GenerateResult

2026-09-25: Result of a generate call.

Functions

generate

2026-09-25: Generate response tokens from a prompt.

generate_speculative

2026-09-25: Generate with MTP speculative decoding, delegated to Model::generate_speculative.

generate_streaming

2026-09-25: Same as generate, but calls on_token(token_id) synchronously after each token, the prefill token included.