generate_streaming

Function generate_streaming 

Source
pub fn generate_streaming<F>(
    model: &dyn Model,
    prompt_tokens: &[u32],
    params: &SamplingParams,
    on_token: F,
) -> Result<GenerateResult>
where F: FnMut(u32),
Expand description

2026-09-25: Same as generate, but calls on_token(token_id) synchronously after each token, the prefill token included.