Module kda 

2026-09-25: Re-exports the Kimi K3 KDA CPU reference, metrale_model_weights::kimi_k3_host::kda.

Decode runs the CUDA launch in super::kda_cuda instead, unless K3_CUDA_KDA=0.

Owner: model-arch, Kimi K3. Invariants: none beyond the types.

Structs

KdaConfig

KDA geometry. Tiny dims are legal for CPU tests; production is 128/4.

KdaState

Per-sequence KDA state. Both buffers are FP32, read-modify-write.

Constants

KDA_L2_EPS

FLA use_qk_l2norm_in_kernel eps. CUDA k3_kda_recurrent_step_f32 uses the same.

Functions

bounded_gate

KDA forget-gate in log space (FLA use_gate_in_kernel).

conv_update

Causal depthwise conv + SiLU. Shifts Metrale-width state left, writes x into the last slot, then y[c] = silu(dot(w[c], state[c])).

cuda_kda_enabled

LinearAttention BoundLayer uses CUDA kda_decode unless K3_CUDA_KDA=0. Projections, AttnRes, and MLP stay on the host either way.

full_rank_output_gate

Full-rank output gate: sigmoid(g) ⊙ RMSNorm(core) per head.

kda_decode_token

One decode token: conv update then recurrent step.

kda_from

Map parsed ModelConfig onto KDA geometry.

kda_recurrent_step

One-token KDA core. qkv is post-conv [3 * qkv_dim] (q|k|v). Updates state.recurrent in place. q/k are L2-normalised here.