generate

Function generate 

Source
pub fn generate(
    model: &dyn Model,
    prompt_tokens: &[u32],
    params: &SamplingParams,
) -> Result<GenerateResult>
Expand description

2026-09-25: Generate response tokens from a prompt.

Runs prefill, takes its token, then decodes until params.max_tokens tokens are out or a token in params.stop_token_ids is produced. The prefill token is always returned, so the output has at least one token.