Struct GemmDispatch 

pub struct GemmDispatch {
    pub fp8_blockscaled_prefill: bool,
    pub cublas: CublasScope,
    pub cublas_fp8: bool,
    pub cutlass_gemm: bool,
    pub cutlass_nvfp4_gemm: bool,
    pub cutlass_nvfp4_qkvz: bool,
    pub cutlass_nvfp4_attn_q: bool,
    pub cutlass_nvfp4_attn_kv: bool,
    pub cutlass_nvfp4_attn_o: bool,
    pub cutlass_nvfp4_ssm_out: bool,
    pub w4a16_variant: u8,
}

2026-09-25: Which GEMM implementation each projection takes, resolved by GemmDispatch::from_env.

Fields

fp8_blockscaled_prefill: bool

2026-09-25: Block-scaled FP8 prefill (per-128-block weight scales and per-token activation scales). On unless METRALE_FP8_SINGLE_SCALE=1.

cublas: CublasScope

2026-09-25: Which projection families take a cuBLASLt GEMM arm (METRALE_CUBLAS_GEMM).

cublas_fp8: bool

2026-09-25: Row-wise FP8 cuBLASLt GEMM for the GDN in_proj_qkvz prefill (METRALE_CUBLAS_FP8=1).

cutlass_gemm: bool

2026-09-25: CUTLASS BF16 GEMM for the GDN in_proj_qkvz prefill, on a memoized FP8→BF16 dequant of the weight (METRALE_CUTLASS_GEMM=1).

cutlass_nvfp4_gemm: bool

2026-09-25: CUTLASS NVFP4 GEMM (METRALE_CUTLASS_NVFP4_GEMM=1). Implies the per-projection NVFP4 flags below except cutlass_nvfp4_ssm_out.

cutlass_nvfp4_qkvz: bool

cutlass_nvfp4_attn_q: bool

cutlass_nvfp4_attn_kv: bool

cutlass_nvfp4_attn_o: bool

cutlass_nvfp4_ssm_out: bool

w4a16_variant: u8

2026-09-25: METRALE_W4A16_VARIANT: v1, v2 and v3 give 1, 2 and 3; any other value, or none, gives 0, which the attention prefill treats like 2.

Implementations

impl GemmDispatch

pub fn from_env() -> Self

2026-09-25: Resolve from the environment, and log the resolved METRALE_CUBLAS_GEMM scope when that variable is set.

pub fn defaults() -> Self

2026-09-25: Everything off except block-scaled FP8 prefill: what an environment with none of these variables resolves to.

pub fn cutlass_nvfp4_attn_qkv(&self, label: &str) -> bool

2026-09-25: Whether CUTLASS NVFP4 is on for an attention projection: q_proj reads the Q flag, k_proj and v_proj the KV flag, and any other label the umbrella flag.

Trait Implementations

impl Clone for GemmDispatch

fn clone(&self) -> GemmDispatch

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GemmDispatch

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for GemmDispatch

fn default() -> Self

Returns the “default value” for a type.

impl PartialEq for GemmDispatch

fn eq(&self, other: &GemmDispatch) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for GemmDispatch

impl Eq for GemmDispatch

impl StructuralPartialEq for GemmDispatch