analyze_bitmask

Function analyze_bitmask 

Source
pub fn analyze_bitmask(mask: &BitmaskSlice<'_>) -> Forced
Expand description

Analyze a fully-computed next-token bitmask for the forced-token condition.

mask MUST be the final mask produced by the matcher’s normal partition path (accepted ∪, rejected ∩, uncertain resolved, stop tokens folded, special tokens cleared) — i.e. exactly the mask the sampling step would apply to the logits. Given that, this returns:

The scan works word-by-word over the packed mask using a hardware popcount, and stops as soon as a word pushes the running set-bit count past one. A genuine choice point — the common case — is therefore rejected within the first non-empty word; it never walks the whole vocabulary bit by bit.