Module quant_weights

Module quant_weights 

Source
Expand description

2026-09-25: The QuantWeights trait: the matvec operations a per-layer forward needs from a quantised weight, so super::qwen3_5 never names a concrete weight type.

Owner: model-layers. Invariants: none beyond the types.

gemv is required. The fused operations are optional: gemv_gate_up_with defaults to two gemv calls, and gemv_silu_gate / gemv_silu_gate_resid default to an error, so a caller learns that no fused kernel exists instead of getting a silent slow path.

Traitsยง

QuantWeights
2026-09-25: A quantised [N, K] weight that runs matvecs on a GpuBackend. The one implementation, for MlxInt8Weight, is in this file.