Module dense_cuda 

2026-09-25: CUDA SiTU-GLU MLP for Kimi K3 (a dense layer’s MLP or a LatentMoE layer’s shared expert) on the resident BF16/FP32 weights, with FP32 activations.

Decode uses it only with K3_CUDA_DENSE=1. Only the input row and the output row cross the host boundary; the weights are read where the loader put them.

Owner: model-arch, Kimi K3. Invariants:

• Every scratch buffer a launch allocates gets a free call when it returns, on success and on error (Scratch on drop); a failed free is ignored.

Functions

launch_dense_matrices

2026-09-25: Runs the gate/up SiTU kernel and the down kernel on explicit device weights. Dtype tags: 0 = FP32, 1 = BF16.

launch_dense_mlp

2026-09-25: Finds the layer’s resident gate/up/down weights (under .mlp. for a dense layer, .block_sparse_moe.shared_experts. for a LatentMoE layer), checks each holds hidden * inter values, and runs launch_dense_matrices.