pub fn allocate_token_bitmask(
batch_size: usize,
vocab_size: usize,
) -> Box<[i32]>Expand description
Allocate a token bitmask: an i32 buffer of
batch_size * ceil(vocab_size/32) words, initialized to all-ones
(-1, i.e. every token allowed).
Port of the vendored matcher::allocate_token_bitmask.