Module vocab_type
metrale_
grammar
1.0.0-beta-preview
Module vocab_
type
Module Items
Enums
In metrale_
grammar::
tokenizer
metrale_grammar
::
tokenizer
Module
vocab_
type
Copy item path
Source
Enums
ยง
Vocab
Type
How a tokenizer encodes its vocabulary strings.