Module engram 

2026-09-25: DeepSeek-V4.1 engram, CPU reference, checked stage by stage against the golden:

1. Hash (NgramHashState): compressed token map, n-gram window with pad and a cache that carries look-back across the prefill/decode split, per-lookback multiply, XOR-rolled, then % prime per (n-gram size, head) plus that slot’s offset. Integer-exact.
2. Table rows (regen_table, embed_rows): the fp8 e4m3 table, dequantised per scale_block values by a power-of-two scale, gathered by hash id. An e4m3 value times a power of two is exact in bf16, so this stage is exact too.
3. Projection (wkv): one bf16 GEMM, [n_hash_cols * head_dim] -> [dim * (hc_mult + 1)], split into one key per hyper-connection copy plus one shared value.
4. Gate (gate_and_add): normalised dot product of stream against key, signed-sqrt sigmoid, h + gate * value, back to bf16.

The hash tables (token_map, multipliers, primes, offsets) are read from the golden, never recomputed, as the loader reads them from the GGUF’s deepseek41.engram.* metadata.

Owner: model-arch, DeepSeek-V4.1 reference. Invariants: none beyond the types.

Structs

EngramTables

2026-09-25: The engram hash tables, geometry and table scales, from the golden’s fixture.engram.

NgramHashState

2026-09-25: The n-gram hash state (the reference’s NgramHashState). The cache holds compressed ids for every position seen, so a decode step’s look-back reaches into the prefill.

Functions

e4m3_to_f32

embed_rows

2026-09-25: Gather rows of a dequantised table by hash id. Output [n_ids, head_dim].

f32_to_e4m3_rne

2026-09-25: Encode to fp8 e4m3fn with round-to-nearest-even. NaN -> 0x7F; above 448 saturates to 448 (0x7E).

f32_to_e8m0_rne

2026-09-25: f32 -> e8m0 as a power of two: nearest exponent, ties (exactly 1.5 * 2^e) to the even exponent. Pinned against the golden’s scales by e8m0_rounding_rule_matches_torch.

gate_and_add

2026-09-25: The engram after the projection: x [tokens, hc, dim] (bf16 values), kv [tokens, dim*(hc+1)], q/k weights [hc, dim]. Returns bf16-rounded [tokens, hc, dim].

linear_bf16

2026-09-25: bf16 linear: f32 accumulate, bf16 output. x: [rows, in], w: [out, in].

regen_bf16_matrix

2026-09-25: A bf16-stored [out, in] weight regenerated by the fixture rule scale = in^-0.5, offset 0.

regen_bf16_qk

2026-09-25: A bf16-stored engram q/k weight of n values regenerated by the fixture rule (offset 1, scale 0.05).

regen_table

2026-09-25: The fixture’s fp8 table for layer index layer_idx, regenerated and dequantised with the golden’s per-block scales (value times scale, then bf16). [rows * head_dim] f32; exact, since e4m3 times a power of two is bf16-exact.