pub fn bitmask_size(vocab_size: usize) -> usize
Number of i32 words needed to hold a bit-per-token mask for a vocabulary of vocab_size tokens.
i32
vocab_size
Port of GetBitmaskSize / DynamicBitset::GetBufferSize: ceil(vocab_size / 32).
GetBitmaskSize
DynamicBitset::GetBufferSize
ceil(vocab_size / 32)