LayerAuxState

metrale_model_layers::layer

Trait LayerAuxState 

Source 
pub trait LayerAuxState {
    // Provided methods
    fn snapshot_aux(
        &self,
        _state: &dyn LayerState,
        _gpu: &dyn GpuBackend,
        _stream: u64,
    ) -> Result<Option<Vec<u8>>> { ... }
    fn has_aux_state(&self) -> bool { ... }
    fn restore_aux(
        &self,
        _state: &mut dyn LayerState,
        _blob: &[u8],
        _gpu: &dyn GpuBackend,
        _stream: u64,
    ) -> Result<()> { ... }
}
Expand description

2026-09-26: A supertrait of TransformerLayer; see the module header.

Provided Methods§

Source

fn snapshot_aux( &self, _state: &dyn LayerState, _gpu: &dyn GpuBackend, _stream: u64, ) -> Result<Option<Vec<u8>>>

2026-09-25: Host-serialized per-sequence state that must travel with an SSM snapshot for a prefix-cache hit to be complete, such as the PLE and QSA carries. Device-to-host copies inside must be ordered on stream. Default None.

Source

fn has_aux_state(&self) -> bool

2026-09-25: True when this layer produces aux state. When any layer does, restore sites decline a snapshot that has no aux blobs (requires_aux_state).

Source

fn restore_aux( &self, _state: &mut dyn LayerState, _blob: &[u8], _gpu: &dyn GpuBackend, _stream: u64, ) -> Result<()>

2026-09-25: Apply a blob from Self::snapshot_aux to this layer’s state (apply_aux_states). The default returns an error.

Implementors§