Expand description
2026-10-01: Default-Tier Boot: a boot-only check of the serve’s memory plan. Once the
serve is up it sends one short warmup wave, then reads GET /memory and reports the KV
pool’s block count and whether the serve’s device footprint stays inside its
--gpu-memory-utilization budget.
Every other gate pins a reduced-precision serve; this one exists for the serve’s default
precision tier, whose KV pool can shrink without any of them noticing. It lands
informational: the metrics are recorded and the verdict is info until bounds are declared
for it in BENCH.toml.
The pins (the benchmark’s definition, not parameters):
- Warmup:
WARMUP_REQUESTSconcurrent chat requests, each a distinct prompt of aboutWARMUP_PROMPT_TOKENStokens withmax_tokens=WARMUP_OUTPUT_TOKENS, temperature 0, seed 0. It brings the serve to the state a first real wave leaves it in. - Read: one
GET /memoryafter the wave has finished.
Owner: bench, default-tier-boot.
Invariants: the run verdict is never a pass (memory::verdict_for).
Re-exports§
pub use memory::Footprint;pub use memory::MemoryReport;pub use memory::footprint;pub use memory::verdict_for;
Modules§
- memory
- 2026-10-01: What a server reports about its memory (
GET /memory), and the pure arithmetic the default-tier boot check derives from it. The server fillsMemoryReport; the driver reads it back, so the one type is the wire contract on both sides.
Structs§
Constants§
- DESCRIPTOR
- MEMORY_
PATH - 2026-10-01: The server’s memory endpoint.
- METADATA