Struct GrammarCompiler 

pub struct GrammarCompiler { /* private fields */ }

Compiler that turns grammars / schemas / structural tags into a CompiledGrammar, bound to a single tokenizer.

Port of the vendored xgrammar::GrammarCompiler.

Implementations

impl GrammarCompiler

pub fn new( tokenizer_info: &TokenizerInfo, max_threads: i32, cache_enabled: bool, cache_limit_bytes: isize, ) -> Result<Self, String>

Construct the compiler.

max_threads and cache_limit_bytes keep the vendored integer types (i32 / isize); they are range-checked and forwarded to the pure-Rust core. Port of the vendored GrammarCompiler::new.

pub fn compile_json_schema( &mut self, schema: &str, any_whitespace: bool, indent: Option<i32>, separators: Option<(impl AsRef<str>, impl AsRef<str>)>, strict_mode: bool, max_whitespace_cnt: Option<i32>, ) -> Result<CompiledGrammar, String>

Compile a JSON schema. Port of the vendored GrammarCompiler::compile_json_schema.

pub fn compile_builtin_json_grammar( &mut self, ) -> Result<CompiledGrammar, String>

Compile the built-in standard-JSON grammar. Port of the vendored GrammarCompiler::compile_builtin_json_grammar.

pub fn compile_grammar_from_ebnf( &mut self, ebnf_string: &str, root_rule_name: &str, ) -> Result<CompiledGrammar, String>

Compile a grammar from an EBNF string. Port of the vendored GrammarCompiler::compile_grammar_from_ebnf.

pub fn compile_grammar( &mut self, grammar: &Grammar, ) -> Result<CompiledGrammar, String>

Compile a Grammar handle. Port of the vendored GrammarCompiler::compile_grammar.

The pure-Rust core consumes a GrammarData by value; the handle is cloned so the caller keeps ownership (vendored took &Grammar).

pub fn compile_structural_tag_json( &mut self, structural_tag_json: &str, ) -> Result<CompiledGrammar, String>

Compile a grammar from a structural-tag JSON document. Port of the vendored GrammarCompiler::compile_structural_tag (raw-JSON form).

pub fn clear_cache(&mut self)

Clear the compiled-grammar cache. Port of the vendored GrammarCompiler::clear_cache.

pub fn get_cache_size_bytes(&self) -> i64

Approximate cache size in bytes. Port of the vendored GrammarCompiler::get_cache_size_bytes.

pub fn load_mask_snapshot(&self, path: &Path) -> Result<usize, String>

Seed the cross-grammar rule-level mask cache from a snapshot a previous process wrote (#918). Returns masks imported; 0 is a miss (absent / stale / corrupt file), never an error.

pub fn save_mask_snapshot( &self, path: &Path, max_entries: usize, ) -> Result<usize, String>

Persist the cross-grammar rule-level mask cache so the next process starts warm (#918). Returns masks written.

pub fn rule_cache_handle(&self) -> Option<RuleLevelCache>

A CLONE of the cross-grammar rule-level cache handle (Arc inside — this shares the cache, it does not copy it).

#918: lets a caller hand the cache to a background thread that persists it, without holding the compiler (which is !Sync).

pub fn snapshot_identity(&self) -> SnapshotIdentity

The identity an on-disk mask snapshot must match to be usable with this compiler’s tokenizer (#918).

pub fn rule_cache_len(&self) -> usize

Number of masks held by the cross-grammar rule-level cache.

pub fn cache_limit_bytes(&self) -> i64

The configured cache memory limit (-1 = unlimited). Port of the vendored GrammarCompiler::cache_limit_bytes.