Module ids

Module ids 

Source
Expand description

2026-09-25: PLE n-gram row ids: the EOS-aware right shift and the multiply-XOR hash.

Owner: model-layers (PLE). Invariants:

  • Pure token-id arithmetic: no device and no weights, so tests.rs checks it bit-exactly against ids recorded from the reference forward.
  • ple_ngram_ids returns one row of ngram_heads() ids per input token.

The LongCat n-gram hash in ngram_embed/ids.rs sums shift * m and takes one modulus. This one multiplies each shifted token by an odd multiplier, XORs the products, then takes a per-head modulus and adds a per-head offset into one concatenated table. Both give valid rows for the same tokens, so using the wrong one fails silently.

Reference: Qwen4ExpTextNGramEmbedding.forward and _shift_right_ignore_eos in bench/qwen4_exp/ref/modeling_qwen4_exp.py.

Structs§

PleIdDims
2026-09-25: Geometry for one PLE site, read from the checkpoint rather than derived.

Functions§

ple_ngram_ids
2026-09-25: Row ids for every head, one row per token in tokens.