Expand description
2026-09-25: The QuantWeights trait: the matvec operations a per-layer forward
needs from a quantised weight, so super::qwen3_5 never names a concrete
weight type.
Owner: model-layers. Invariants: none beyond the types.
gemv is required. The fused operations are optional: gemv_gate_up_with
defaults to two gemv calls, and gemv_silu_gate / gemv_silu_gate_resid
default to an error, so a caller learns that no fused kernel exists instead
of getting a silent slow path.
Traitsยง
- Quant
Weights - 2026-09-25: A quantised
[N, K]weight that runs matvecs on aGpuBackend. The one implementation, forMlxInt8Weight, is in this file.