Expand description
2026-09-25: PLE n-gram row ids: the EOS-aware right shift and the multiply-XOR hash.
Owner: model-layers (PLE). Invariants:
- Pure token-id arithmetic: no device and no weights, so
tests.rschecks it bit-exactly against ids recorded from the reference forward. ple_ngram_idsreturns one row ofngram_heads()ids per input token.
The LongCat n-gram hash in ngram_embed/ids.rs sums shift * m and takes
one modulus. This one multiplies each shifted token by an odd multiplier,
XORs the products, then takes a per-head modulus and adds a per-head offset
into one concatenated table. Both give valid rows for the same tokens, so
using the wrong one fails silently.
Reference: Qwen4ExpTextNGramEmbedding.forward and
_shift_right_ignore_eos in bench/qwen4_exp/ref/modeling_qwen4_exp.py.
Structs§
- PleId
Dims - 2026-09-25: Geometry for one PLE site, read from the checkpoint rather than derived.
Functions§
- ple_
ngram_ ids - 2026-09-25: Row ids for every head, one row per token in
tokens.