Module lexer

Module lexer 

Source

Structs§

EbnfLexer
The EBNF lexer. Equivalent to xgrammar’s EBNFLexer::Impl.
LexError
A lexer error with source position.
Token
A single lexer token. Port of EBNFLexer::Token.

Enums§

TokenType
EBNF token kinds. Port of EBNFLexer::TokenType.
TokenValue
The processed value carried by a Token.