bounded_gate

Function bounded_gate 

Source
pub fn bounded_gate(
    g_lowrank: &[f32],
    dt_bias: &[f32],
    a_log: &[f32],
    dims: KdaDims,
    lower_bound: f32,
) -> Vec<f32>
Expand description

2026-09-25: The bounded GLM forget gate:

gate[t, h, d] = lower_bound * sigmoid(exp(a_log[h]) * (g_lowrank[t, h*D + d] + dt_bias[h*D + d]))

a_log is per head and dt_bias per channel, so they are separate arguments.