Struct SamplingParams 

pub struct SamplingParams {

    pub temperature: f32,
    pub top_k: u32,
    pub top_p: f32,
    pub top_n_sigma: f32,
    pub min_p: f32,
    pub logit_bias: Vec<(u32, f32)>,
    pub repetition_penalty: f32,
    pub repetition_penalty_window: u32,
    pub presence_penalty: f32,
    pub frequency_penalty: f32,
    pub lz_penalty: f32,
    pub dry_multiplier: f32,
    pub dry_base: f32,
    pub dry_allowed_length: u32,
    pub dry_sequence_breakers: Vec<u32>,
    pub max_tokens: usize,
    pub stop_token_ids: Vec<u32>,
    pub seed: Option<u64>,
}

2026-09-25: Sampling parameters for a request.

Fields

temperature: f32

2026-09-25: Temperature. 0.0 is greedy (is_greedy).

top_k: u32

2026-09-25: Top-k: keep only the k highest-probability tokens. 0 disables it.

top_p: f32

2026-09-25: Top-p (nucleus): keep the smallest set of tokens whose cumulative probability reaches p. 1.0 or more disables it.

top_n_sigma: f32

2026-09-25: Top-n-sigma: before temperature scaling, drop tokens whose logit is below mean - n*sigma. 0.0 disables it.

min_p: f32

2026-09-25: Min-p: keep tokens with prob >= min_p * max_prob (post-softmax). 0.0 disables it.

logit_bias: Vec<(u32, f32)>

2026-09-25: Per-token logit bias as (token_id, bias) pairs, added after the penalties and before any filter.

repetition_penalty: f32

2026-09-25: Repetition penalty: for every occurrence of a token in the window, a positive logit is divided by it and a non-positive one multiplied. 1.0 and values <= 0.0 disable it.

repetition_penalty_window: u32

2026-09-25: Window, in tokens, for the repetition, presence and frequency penalties. 0 uses the whole history.

presence_penalty: f32

2026-09-25: Presence penalty: subtracted once from the logit of each token that appears in the window. 0.0 disables it.

frequency_penalty: f32

2026-09-25: Frequency penalty: subtracted from a token’s logit once per occurrence in the window. 0.0 disables it.

lz_penalty: f32

2026-09-25: Strength of apply_lz_penalty. Values <= 0.0 disable it.

dry_multiplier: f32

2026-09-25: Multiplier of the DRY (“don’t repeat yourself”) penalty, apply_dry_penalty. Values <= 0.0 disable it.

dry_base: f32

2026-09-25: DRY base: penalty = multiplier * base^(match_len - allowed_len).

dry_allowed_length: u32

2026-09-25: DRY allowed length: only matches longer than this are penalised.

dry_sequence_breakers: Vec<u32>

2026-09-25: DRY sequence breakers: token ids at which a match stops.

max_tokens: usize

2026-09-25: Maximum tokens to generate.

stop_token_ids: Vec<u32>

2026-09-25: Stop token ids.

seed: Option<u64>

2026-09-25: RNG seed. Some seeds a StdRng for each draw; None uses the thread RNG.

Implementations

impl SamplingParams

pub fn greedy(max_tokens: usize) -> Self

2026-09-25: Greedy sampling with every filter and penalty neutral.

pub fn is_greedy(&self) -> bool

Trait Implementations

impl Clone for SamplingParams

fn clone(&self) -> SamplingParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SamplingParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.