Glm5NextDsaWeights

metrale_model_arch::glm5next_dsa::layer

Struct Glm5NextDsaWeights

pub struct Glm5NextDsaWeights {Show 13 fields
    pub q_a_proj: DevicePtr,
    pub q_a_layernorm: DevicePtr,
    pub q_absorb: DevicePtr,
    pub kv_a_proj: DevicePtr,
    pub kv_a_layernorm: DevicePtr,
    pub o_absorb: DevicePtr,
    pub wk: DevicePtr,
    pub k_norm_weight: DevicePtr,
    pub k_norm_bias: DevicePtr,
    pub compress_gate: DevicePtr,
    pub wq_b: DevicePtr,
    pub weights_proj: DevicePtr,
    pub ape: DevicePtr,
}

Expand description

2026-09-25: One DSA block’s weights on the device, already sharded for this rank.

Fields§

§q_a_proj: DevicePtr§q_a_layernorm: DevicePtr§q_absorb: DevicePtr

2026-09-25: [local_heads * kv_lora_rank, q_lora_rank] BF16: q_b_proj absorbed through kv_b_proj’s K half, so Q arrives in latent space.

§kv_a_proj: DevicePtr§kv_a_layernorm: DevicePtr§o_absorb: DevicePtr

2026-09-25: [hidden, local_heads * kv_lora_rank] BF16, row-parallel; the caller all-reduces the output.

o_proj with kv_b_proj’s V half folded in (build::absorb_o), because the decode kernel’s output is in latent space, kv_lora_rank per head. The checkpoint o_proj is v_head_dim per head (256 against 512 on GLM-5.3).

§wk: DevicePtr§k_norm_weight: DevicePtr§k_norm_bias: DevicePtr§compress_gate: DevicePtr§wq_b: DevicePtr§weights_proj: DevicePtr

2026-09-25: Multiplied by index_heads^-0.5 at load; dsa_index_scores does not apply it.

§ape: DevicePtr

2026-09-25: [index_kpool, index_head_dim] FP32; the checkpoint stores BF16.

Auto Trait Implementations§

impl Freeze for Glm5NextDsaWeights

impl RefUnwindSafe for Glm5NextDsaWeights

impl Send for Glm5NextDsaWeights

impl Sync for Glm5NextDsaWeights

impl Unpin for Glm5NextDsaWeights

impl UnwindSafe for Glm5NextDsaWeights

Blanket Implementations§

impl<T> Any for T
where T: 'static + ?Sized,

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more

impl<T> Borrow<T> for T
where T: ?Sized,

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more

impl<T> BorrowMut<T> for T
where T: ?Sized,

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more

impl<T> From<T> for T

fn from(t: T) -> T

Returns the argument unchanged.

impl<T> Instrument for T

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more

impl<T, U> Into<U> for T
where U: From<T>,

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

impl<T> Same for T

type Output = T

Should always be Self

impl<T, U> TryFrom<U> for T
where U: Into<T>,

type Error = Infallible

The type returned in the event of a conversion error.

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

fn vzip(self) -> V

impl<T> WithSubscriber for T

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,