nvfp4_mmq_gemm_tiled

Function nvfp4_mmq_gemm_tiled 

Source
pub fn nvfp4_mmq_gemm_tiled(
    gpu: &dyn GpuBackend,
    kernel_nc: KernelHandle,
    kernel_wc: KernelHandle,
    mmq_x: u32,
    a_fp4: DevicePtr,
    w_nvfp4: DevicePtr,
    out_bf16: DevicePtr,
    m: u32,
    n: u32,
    k: u32,
    out_scale: Option<f32>,
    stream: u64,
) -> Result<bool>
Expand description

2026-09-25: nvfp4_mmq_gemm with the M tile mmq_x chosen by the caller. 2026-09-28: With out_scale, the pipelined 128-row tile applies it in its store and the call returns true; otherwise the caller still owes nvfp4_scale_bf16. The tile choice: the kernels must be that tile’s entries. nvfp4_mmq.cu instantiates 16, 32, 64 and 128. A 128-wide tile issues MMAs for all 128 columns whatever m is, so a small m wants a small tile. grid.y = ceil(m / mmq_x), and each M tile streams the whole weight again, so m must not exceed mmq_x (debug-asserted).