Workspace Layout

Metrale Engine is a twenty-three-member Cargo workspace plus a build-time kernel tree (count them in the root Cargo.toml members list). This chapter maps every top-level directory to its role, and the crates to the axes of variation they each insulate.

Repository tree (top level)

metrale-inference/
├── README.md
├── QUICKSTART.md                 per-model Docker recipes
├── CONTRIBUTING.md, AGENTS.md    contributor workflow
├── SECURITY.md                   disclosure
├── LICENSE-MIT, LICENSE-APACHE   MIT OR Apache-2.0
├── Cargo.toml                    workspace root (21 members)
├── Cargo.lock
├── rust-toolchain.toml           pins the Rust release
├── deny.toml                     cargo-deny allow/deny lists
├── crates/                       Rust source for every crate
├── kernels/                      CUDA source, organized as (hw, model, quant)
├── docker/                       per-hardware Dockerfiles
├── scripts/                      bench, model-sweep, release helpers
├── tests/                        cross-crate integration tests (run_all_models.py lives here)
├── docs/                         design notes, history, release notes
├── jinja-templates/              chat templates for models that need custom ones
├── bench/                        benchmark harnesses and tracked results
├── governance/                   PR journey ledgers (governance/pr-<n>.jsonl)
├── book/                         this book (mdBook source)
└── vendor/                       vendored deps (cudarc)

The workspace members

Cargo.toml lists:

members = [
    "crates/core",
    "crates/config",
    "crates/closure",
    "crates/circuit",
    "crates/governance",
    "crates/kernels",
    "crates/kernel-tree",
    "crates/gpu-sys",
    "crates/telemetry",
    "crates/gpu-runtime",
    "crates/scheduler",
    "crates/cache",
    "crates/sampling",
    "crates/storage",
    "crates/comm",
    "crates/grammar",
    "crates/model-weights",
    "crates/model-layers",
    "crates/model-arch",
    "crates/model-engine",
    "crates/speculative",
    "crates/bench",
    "crates/server",
]

Each is its own crate with its own Cargo.toml, its own unit tests, and its own responsibility:

The dependency graph runs strictly downward in the table above — metrale-core has no internal deps, every crate above it builds on crates below. There are no cycles.

The kernel tree

kernels/
├── gb10/                                        # One directory per hardware target
│   ├── HARDWARE.toml                            # vendor, arch, memory specs, serving defaults
│   ├── common/                                  # the shared baseline every GB10 target compiles
│   ├── qwen3-next-80b-a3b/                      # One directory per model target
│   │   ├── MODEL.toml                           # layer counts, sampling presets, behavior
│   │   └── nvfp4/                               # One directory per quantization target
│   │       ├── KERNEL.toml                      # compile flags, module names, [sources], [shadow]
│   │       └── *.cu                             # the kernels this target overrides
│   ├── qwen3.6-35b-a3b/
│   │   └── nvfp4/
│   ├── mistral-small-4/
│   │   └── nvfp4/
│   └── ... (one leaf per (model, quant) target)
├── hopper/, b200/                               # inherit gb10 ([hardware] inherits)
├── b300/                                        # Kimi K3 bring-up
└── strix/, strix-hip/, metal/                   # AMD and Apple targets

A leaf directory holds only what its (gb10, model, quant) target changes relative to common/; everything else comes from common/. A file in a leaf can use any tile shape, any register budget, any shared-memory layout without affecting a different target, because nothing else compiles it. Philosophy: AI Kernel HyperCompiling describes the layers, [sources] use and [shadow].

This is the mechanism that makes kernels/ a scalable structure. Adding a new GPU is kernels/<new-hw>/. Adding a new model is kernels/<hw>/<new-model>/. Adding a new quantization is kernels/<hw>/<model>/<new-quant>/. Nothing else moves.

Docker layout

docker/
├── gb10/
│   ├── Dockerfile                         multi-model image — compiles every target
│   ├── Dockerfile.builder                 build sandbox with CUTLASS and FlashInfer pinned
│   ├── qwen3-next-80b-a3b/nvfp4/          per-model slim image
│   ├── qwen3.5-35b-a3b/nvfp4/
│   └── ... (one slim Dockerfile per supported model)
├── hopper/, b200/                         H100/H200 and B200 images
├── k3/                                    Kimi K3 bring-up image
└── docker-guide.md                        build + run instructions

The multi-model Dockerfile at docker/gb10/Dockerfile is what ships as metrale/metrale-inference-gb10:latest. Per-model Dockerfiles exist for operators who want a smaller image containing only one target — the kernel registry still uses KernelTarget at runtime, but only one target set is baked in.

Docs, design records, history, releases

Inside docs/:

• adr/ — architecture decision records (licensing, pure-Rust, hybrid SSM/attention, NVFP4/FP8 quantization, TP/EP composition, EP batched decode, etc.). Treat these as the long-form rationale behind code changes; commit messages are deliberately terse and point here. Top-level notes like ARCHITECTURE.md, METRALE_KERNELS.md, and HARDWARE.md sit alongside them.
• METRALE_JOURNEY.md — benchmark journey and retrospective of the engine on GB10. Useful context, but not a contract.
• releases/ — human-readable release notes keyed by release (README.md plus per-release files).

The book you're reading in book/ synthesises all of this into a single narrative — it is not a canonical rewrite of those documents. The design records in docs/adr/ remain the authoritative reference and the book links to them directly from the deep-dive chapters.

What changes when you add a…

Each row touches a small, bounded set of files. That bounded-ness is the architectural payoff of the workspace being split along axes of variation. Read Kernel Dispatch Pipeline next to see the runtime side, or SBIO to see how the trait layering makes the whole thing testable without a GPU.