November 6, 2026 at 11:27 AM CST · charter and measurements: James Meadlock with Milo (one session on anthropic/claude-fable-5-1) · writing and diagrams: one headless session on anthropic/claude-fable-5.1 · one DGX Station GB300 (+ RTX PRO 6000 for recipe B)

Can One GB300 Station Run DeepSeek-V4.1-Flash in 5 ms a Step?

A charter, plus the first two days. Our standalone runtime decodes the real model on the Station with its arithmetic kept; the project's own instrument — the 6-row DSpark step, drafts verified by exact greedy match — now exists and reads 18.6 ms a step, 6.4 ms a token (17.9 and 6.2 in steady state; 20.3 and 7.0 at the end of day one) (production: 7.36 ms a step, about 2.6 ms a token). The 5.0 ms bar turned out to be a bytes bar, and the bar is now: beat 7.36 and publish ms per token (section 8).

Created · Updated

CORRECTION · NOVEMBER 9 This page treats our standalone runtime as the road to a faster single-GPU step. Measured against what we already had, it is not. The runtime reads 156–162 tokens a second on its own test. The single-GPU vLLM recipe we published in October reads 188–219 on its own, different, test. We never ran the two side by side, so no ratio is claimed, but nothing measured supports serving from this runtime. It is now labelled a research instrument: a deterministic reference to check other engines against, and a bench for measuring kernels. The serving question moved to Garner McCloud's engine with its second GPU removed, and that result, measured against both of our existing recipes on one harness, is in DeepSeek-V4.1-Flash on One GB300 Without the Second GPU. The sections below are kept as written.
IN PLAIN ENGLISH We serve DeepSeek-V4.1-Flash on one DGX Station GB300, and today each decode step takes 7.36 ms. We want it at or under 5.0 ms on the same box, with stock weights, all 384 experts, and no KV-cache quantization. The obvious move is to fuse the ~1,300 small kernels per step into fewer, bigger ones, but our arithmetic says that pool is worth about 0.7–1.4 ms on the old 11 ms step; the larger cost is the GPU waiting while a second card computes the cold experts and ships the answer back over PCIe. So the bet is a standalone decode runtime, no vLLM and no SGLang, whose fused MoE layer reads the cold expert rows it needs straight out of Grace RAM over NVLink-C2C, inside the kernel, no sidecar round trip. The number is honest about what has to go. We will also build the sidecar version on the same runtime and publish whichever loses. The kill rule is fixed: a working runtime by December 18, 2026, or we restore today's lane and park the project as desk research. First two days: the runtime exists, decodes the real model, and now runs the real instrument — the 6-row speculative step — at 18.6 ms a step and 6.4 ms a token (17.9 and 6.2 in steady state) against production's 7.66 and 2.55 on the same prompts (sections 8, 8.4). The 1-row step is 9.8 ms (section 8.3). The arithmetic is the model's at every rung but three, and those are within one bf16 ulp.

1. The machine and the model

One GPU with 250.7 GiB of visible HBM, a Grace CPU with 494 GiB of RAM, NVLink-C2C between them; pinned host reads measure around 340 GB/s. DeepSeek-V4.1-Flash-1017 is a 552B-parameter MoE plus 196B of Engram tables, about 475 GiB on disk: 40 MoE layers × 384 MXFP4 experts × 17.93 MiB = 269.0 GiB of expert weights, plus two Engram tables of 94.6 GiB. That cannot all fit HBM (fact, verified 2026-09-10): Engram lives in pinned Grace RAM, and so do some experts. An RTX PRO 6000 (96 GB) sits on PCIe with no P2P path to the big GPU; its results come back through host memory.

Two memories, one model: GB300 HBM 250.7 GiB and Grace RAM 494 GiB joined by NVLink-C2C, with an RTX PRO 6000 on PCIe for recipe B onlyLeft box, GB300 HBM 250.7 GiB holding hot experts, attention and dense weights, and the KV cache. Right box, Grace RAM 494 GiB holding Engram 2 × 94.6 GiB and the cold experts. Two arrows between them labeled NVLink-C2C about 340 GB/s pinned. A dashed RTX PRO 6000 96 GB box hangs off the GPU over PCIe, recipe B only. Conclusion: 269 GiB of experts plus 189 GiB of Engram cannot fit 250.7 GiB of HBM, so some of the model lives in Grace.TWO MEMORIES, ONE MODELGB300 HBM · 250.7 GiBhot expertsattention + denseKV cacheGPU sideGrace RAM · 494 GiBEngram 2 × 94.6 GiBcold expertsCPU side, pinned, no swapNVLink-C2C~340 GB/s pinnedPCIe, no P2PRTX PRO 6000 · 96 GBrecipe B only269 GiB of experts + 189 GiB of Engram cannot fit 250.7 GiB of HBM:some of the model lives in Grace.

Figure 1 — Where the bytes can go. HBM and Grace are facts of the hardware; the ~340 GB/s is a measured pinned-read figure from our notes. The HBM box lists what lives there without a byte total, because we have not published one.

2. Where 11 ms goes

The only full per-role anatomy we have is from the earlier v27 build (2026-10-27), traced with nsys at roughly 0% overhead: 11.216 ms per step, one user, 6 rows per step (DSpark speculative decoding, k=5). GPU 98.6% busy, 0.15 ms idle. 1,305 kernel launches per step; 1,035 ran under 10 µs yet together cost 4.54 ms, 40% of the step. Counters: SMs active 49%, SM issue 6%, tensor 2.3%, DRAM read 32%. Our reading: latency- and occupancy-bound on many tiny kernels; not compute-bound, not launch-bound (CUDA graphs already hide the launches).

Today's lane, Pg61 (vLLM v0.31.0, Garner McCloud's public dgx-station-recipes, our patches), runs the same decode at 7.36 ms (measured, release round 2026-10-31). It has not been profiled per role; that profile is the project's first job.

Where the v27 step went: 11.216 ms measured with nsys, split by kernel roleOne stacked horizontal bar: MegaMoE 3.38 ms, mHC 1.80 ms, attention 1.46 ms, ll_gemm 1.34 ms, sidecar wait 1.33 ms, lm_head 0.42 ms, other 1.34 ms, idle 0.15 ms. A thin bar below shows 1,305 launches per step, 1,035 of them under 10 µs summing to 4.54 ms, 40 percent. Conclusion: GPU 98.6 percent busy, not launch-bound, not compute-bound, many tiny kernels plus a wait on the sidecar.WHERE THE v27 STEP WENT — 11.216 ms, MEASURED WITH nsys0 ms24681011.216 msMegaMoE3.38 msmHC1.80 msattention1.46 msll_gemm1.34 mssidecar wait1.33 msother1.34 mslm_head 0.42 msidle 0.15 msKernel launches per step: 1,3051,035 of them run under 10 µs and together take 4.54 ms (40% of the step)GPU 98.6% busy. Not launch-bound, not compute-bound:many tiny kernels plus a wait on the sidecar.

Figure 2 — The v27 step by role (measured, nsys, 2026-10-27). "Other" is 11.216 ms minus the listed roles and idle: 1.34 ms. "Sidecar wait" is the _finish2 span where the GPU has finished its hot-expert math and is waiting for the RTX card's rows to come back over PCIe. Pg61's 7.36 ms step has no equivalent breakdown yet.

Start with what a perfect megakernel cannot remove. Bojie Li's sm_120 microbenchmarks (author's claim, 2026-09-26) put a PDL one-to-one handoff at about 0.37 µs and an all-SM gather at about 1.0 µs; at roughly 1,370 dependency boundaries per step, that is 0.5–1.4 ms of irreducible cost even inside one kernel. Our outside-view model (W1, 2026-10-28) prices everything fused-layer or PDL work can address at 50–70 µs per layer: 2.0–2.8 ms theoretical, −0.7 to −1.4 ms realistic, 6–13% of the 11.0 ms v27 step. Pg61 has already spent some of that pool (it inherits the peer's PDL and fused-sampler tree), so it cannot be subtracted again from 7.36 ms.

Peer evidence agrees (author's claims, 2026-10-28): a PDL reconstruction of the Hazy megakernel reaches 97.3% of a single persistent kernel with 81 ordinary kernels; SGLang #46088 found a fused Mega-mHC slower at every batch size; vLLM-Moet measured a +2.7% step regression from fusing routing into the MoE kernel. Nobody has published one kernel per forward with a PCIe sidecar, host Engram, sparse indexer and speculative decoding in the loop.

The peers who are fast use a chain. NVIDIA's sgl-nvidia-mega SGLang plugin (PR #1, merged 2026-10-19) decodes DeepSeek-V4.1-Flash at a p50 of 3.386 ms per step, batch 1, 6 rows, on four GB300s in TP4: 10–15 right-sized kernels per layer handing outputs through mailboxes, PDL and side streams. Per-GPU weight bytes there are a quarter of ours and no experts live off-GPU; that number is evidence for the shape, not a target for one Station. vLLM's specialized decode for Kimi K3 (2026-10-16) is the same shape: 23 kernels → 13 per layer, 130 µs → 69 µs, on GB300 at TP8/TP16.

So the project (first chartered under the banner "megakernel", renamed the same day) is defined as whatever makes the decode step shortest: first a fused multi-kernel chain with mailboxes and PDL, one persistent kernel per decoder layer as the stretch, one kernel per forward not a goal. The vehicle is a standalone decode runtime: one Python process, no vLLM, no SGLang; Python host, CuTe DSL kernels, Triton glue, CUDA C++ for the scheduler and mailboxes. Only decode gets optimized; prefill speed is a non-goal until the headline is met. Milestone 0 borrows every existing kernel as a library (DeepGEMM MegaMoE, FlashMLA sparse attention, FlashInfer MXFP4 MoE and MXFP8 GEMMs, vLLM's Engram gather, the peer's draft samplers) and replaces a piece only when the step trace names it and the replacement is measured against it in the same window.

What fusion can and cannot buy: v27 at 11.2 ms with a modeled fusion pool of 0.7 to 1.4 ms, Pg61 at 7.36 ms, and the 5.0 ms target lineTwo horizontal bars on a 0 to 12 ms axis. The v27 bar reaches 11.2 ms and its last 0.7 to 1.4 ms is hatched as the modeled fusion pool. The Pg61 bar reaches 7.36 ms. A dashed vertical line marks the 5.0 ms target. Conclusion: fusion alone does not reach 5.0 ms; the sidecar round trip has to go.WHAT FUSION CAN AND CANNOT BUYfusion pool, modeled: −0.7…−1.4 msv27 step 11.2 msv27 (measured)Pg61 today 7.36 msPg61 (measured)024681012 mstarget ≤ 5.0 msFusion alone does not reach 5.0 ms. The sidecar round trip has to go.

Figure 3 — The fusion pool is modeled (W1, 2026-10-28) against the measured v27 step; the hatched slice is the realistic −0.7 to −1.4 ms. Pg61 is measured. The target is a decision. The gap between Pg61 and the line is larger than anything fusion is modeled to buy.

4. The two recipes

Why the round trip costs what it does: per layer at 6 rows, the hot-expert kernel touches about 25 unique experts, roughly 448 MiB, about 65 µs at ~7 TB/s of HBM (modeled, W1; we measured ~81 µs). Then the GPU waits on the RTX card, a ~112 µs MoE section on about 80% of layers: the 1.33 ms _finish2 wait on v27. The hot math is not the problem; the waiting is.

Recipe A comes first and is Grace-only. The thesis: a fused layer gathers the cold expert rows it needs directly from pinned Grace RAM over C2C, inside the kernel. One memory system, no sidecar, no GPU spinning on a PCIe mailbox. Recipe B is the same runtime plus a persistent cooperative kernel on the RTX PRO 6000: the peer's "GP1" shape, the configuration we know is fast. If A's C2C self-fetch loses to B on measurement, that is a published result.

Recipe A versus recipe B for one MoE layer: Grace-only C2C self-fetch versus an RTX PRO 6000 sidecar over PCIeTwo panels. Recipe A, first, Grace-only: a GPU box running the fused layer kernel exchanges rows with HBM hot expert rows below it and with pinned Grace RAM cold expert rows over C2C to its right; caption one memory system, no spin. Recipe B, sidecar: the same GPU box exchanges rows with HBM and sends work over PCIe to an RTX PRO 6000 whose persistent cooperative kernel computes cold experts; caption the peer's GP1 shape. Conclusion: A tests the thesis, B is the known-fast configuration, whichever loses is a published result.RECIPE A vs RECIPE B — ONE MoE LAYERRecipe A (first): Grace-onlyGPUfused layer kernelGrace RAM (pinned)cold expert rowsC2CHBMhot expert rowsone memory system, no spinRecipe B: sidecarGPUfused layer kernelRTX PRO 6000persistent cooperativekernel computescold expertsPCIeHBMhot expert rowsthe peer's GP1 shapeA tests the thesis; B is the known-fast configuration.Whichever loses is a published result.

Figure 4 — One MoE layer under each recipe. In both, the hot expert rows are in HBM and Engram stays in Grace. The difference is where the cold experts are computed: inside the GPU's own kernel from C2C-reachable Grace memory (A), or on a second card that answers over PCIe (B).

5. The number and the ladder

The headline metric is single-user decode step time at 6 rows, measured with nsys on the cudaProfilerApi range over a fixed prompt corpus. Baseline: Pg61, 7.36 ms. Bar, restated November 7, 2026: beat Pg61 — under 7.36 ms per 6-row step — and publish ms per token alongside it. The charter's original bar was ≤ 5.0 ms per step, chosen to exceed the modeled fusion pool on purpose; the first two days showed it is a bytes bar the box cannot meet with stock weights (section 8), and James restated it. 5.0 stays on record as the original.

MTargetDone when
W0first dark window, ~1 dayPg61 per-role nsys step breakdown at 7.36 ms; per-boundary PDL gap histogram; 1-row greedy logit capture as the parity reference. Taken while Pg61 still exists; then Pg61 goes down. 2026-11-06: parity capture and step cell done; the nsys breakdown is blocked (see Status).
M0Dec 18, 2026 → met Nov 7 under the approved parity barStandalone runtime, borrowed kernels end to end, greedy 1-row parity vs the W0 capture, 32K context, step trace with named boundaries, step ≤ Pg61 + 10% at 1-row-equivalent. 2026-11-07: runtime decodes the real model, one CUDA graph per step, 10.5 ms at 1 row (section 7); parity bar under amendment.
M1Jan 2027 → done Nov 7DSpark k=5 in the runtime, 6-row step, headline instrument live → first honest ms/step vs 7.36. 2026-11-07: instrument live; 20.3 ms a step, 2.90 tokens a step, 7.0 ms a token by day's end, the 6-row step bit-identical to the 1-row step (sections 8, 8.1); 18.7 ms a step and 6.5 ms a token by the end of the night — residency, the small wins, staging overlapped with compute, the hot scale factors no longer re-copied; MegaMoE measured and declined (section 8.3). 2026-11-08: production measured on the same prompts (7.66); headline and steady-state columns; 18.6 / 6.4 and 17.9 / 6.2 (section 8.4).
M2Feb 2027Recipe A fused chain + C2C self-fetch of cold rows — the thesis test; plus an "everything in Grace" bounding measurement.
M3Mar 2027Recipe B: persistent cooperative RTX kernel on the same runtime.
M4—Minimal OpenAI-compatible surface; 128K → 1M context; production-candidacy review.

Month labels are targets, not promises. Context ladder is 32K → 128K → 1M; 1M gates production candidacy only. Rows C1–C8 are designed in; C32 is a declared non-goal of the runtime.

Kill rule: milestone 0 by December 18, 2026 (six weeks), or Pg61 is restored and the project parks as desk research.

6. What stays fixed

Stock weights: no NVFP4, no requantization. No KV-cache quantization. No expert pruning, no router masking; all 384 experts stay. Quality gates as calibrated. Determinism is a design requirement: bit-identical output across runs and relaunches, fixed-order reductions, no atomics on the data. Today's sidecar lane fails that (measured: 79,376 of 100,000 sidecar calls differ between two runs of the same binary; reduction order depends on arrival when two or more cold experts are summed). The one thing reopened is layout: weight tiling on disk and in HBM may change, as long as every value stays bit-identical to the pinned checkpoint (HF revision 1bae0e10).

7. Status today

Day one and the night after (November 6–7, 2026). The standalone runtime exists and decodes the real checkpoint on one GB300: DeepSeek's own reference model.py, unchanged, on our tiered loader (hot experts and everything dense in HBM, 218.6 GiB; cold experts and the Engram tables in pinned Grace RAM, read over NVLink-C2C from inside the kernels). Boot is about 60 seconds. The 1-row decode step went 404.9 → 13.9 ms in a day and a night; at 13.9 ms the GPU is busy for 13.95 ms of it, so there is no launch overhead left, only kernel time (2,827 kernels per step, down from 20,550). Every rung was measured with the same instrument and graded against the rung before it: five of the nine are bit-identical (every fp32 logit hashed at every position, 32 steps × 8 prompts, replay and relaunch, memory fills on and off), the other four are kernels we borrowed or wrote that reproduce the reference within one bf16 ulp on a small fraction of outputs. The exact path stays selectable for parity runs.

Decode step time of our runtime, 1 row, measured on the GB300 on November 6 and 7, 2026: 404.9, 98.4, 44.6, 32.2, 23.8, 19.9 and 13.9 ms, against production vLLM at 7.36 ms (6 rows) and the 5.0 ms barHorizontal bars on a log scale, one per rung; blue bars are bit-identical to the rung before, green bars are within one bf16 ulp. Dashed reference lines at 7.36 ms (production, 6 rows, a different instrument) and 5.0 ms (the bar). Our runtime's decode step, 1 row (measured, cuda events, steps 32–512) log scale · blue = bit-identical to the rung before · green = within one bf16 ulp on ≤ 0.04 % of outputs reference forward, eagerfirst light · 20,550 launches404.9 msone CUDA graph per stepT8 · bit-identical98.4 msgrouped expert GEMMT9a-1 · DeepGEMM · 1-ulp class44.6 msMXFP8 GEMV, 290 projectionsT9b-1 · 1-ulp class32.2 msseed map · glue I · bounded attnA2-1, T9b-2, T9b-3a · bit-identical23.8 msFlashMLA sparse attentionT9b-3c · 1-ulp class19.9 msin-place experts · glue IIT9a-2, T9b-2 R2 · bit-identical13.9 ms 1 ms10 ms100 ms1000 ms production vLLM 7.36 (6 rows, other instrument) bar 5.0 The two reference lines are the target, not a comparison: a 1-row step against a 6-row speculative step. The 6-row instrument arrives at milestone 1.

Figure 5 — The first 36 hours of the runtime. Measured on the Station (cuda events over steps 32–512 of a 512-token greedy generation, 1 row). Receipts: docs/first-light-*.json and docs/NIGHT-LADDER-20261107.md in the repo.

RungWhat changedms/stepArithmetic
First lightReference forward as shipped: 20,550 launches per step, GPU busy 109 ms of 405; the reference MoE does a host sync and a 384-iteration Python loop in every layer.404.9bit-identical self-replay
T8One CUDA graph per step. The MoE loop became device routing plus one gather through a per-expert pointer table into a static buffer, so cold experts are read over C2C by the same kernel.98.4bit-identical
T9a-1The 18 per-expert TileLang launches per layer became 3 DeepGEMM masked grouped GEMMs consuming the checkpoint's bytes as-is.44.61-ulp class
T9b-1A Triton MXFP8 GEMV (split-K, tensor-core dot, per-shape tuned) for the 290 small fp8 projections: 15.8 → 3.4 ms.32.21-ulp class
A2-1The hot-expert map from production's routing statistics instead of "the first 295 ids": cold calls 23.5 % → 0.9 % of routed experts. Residency changes nothing in the arithmetic by construction.30.9byte-identical output
T9b-2 R1Seven zero-risk glue clusters fused (SwiGLU + quantize, combine, mHC post, RoPE, six gathers into one): −2,080 launches.26.0bit-identical
T9b-3aThe reference sparse-attention kernel bounded to the valid index blocks (the padded −1 blocks are provably no-ops), and the expert gather rewritten to 16-byte lanes: 1.3 → 2.9 TB/s.23.8bit-identical
T9b-3cFlashMLA's sparse-prefill op, loaded from the vLLM wheel's shared library without importing vLLM, reading our bf16 caches as they are: 125 → 9 µs per call. Its fp64 error is 1.04× the reference kernel's own.19.91-ulp class
T9a-2DeepGEMM's contiguous grouped GEMM reading hot experts straight from the pool (no staging copy); only cold experts are staged. The pool keeps the checkpoint's e8m0 scales — the kernel's packed scale format is those bytes regrouped.17.8bit-identical to the masked path
T9b-2 R2Seven medium-risk clusters fused (RMSNorm statistics, router, kv/q paths, mHC mixes, Engram, indexer post-processing): each kernel's reduction order was calibrated on the device against torch on 1,024 random rows and the matching order chosen; a cluster that cannot match is left unfused. torch's topk is kept for its tie order. −2,822 launches.13.9bit-identical

All figures measured on the Station on November 6–7, 2026 (cuda events, steps 32–512, 1 row, greedy). Not comparable to the 7.36 ms production number, which is a 6-row speculative step on a different instrument; the project's headline instrument arrives at milestone 1.

Where the remaining 13.9 ms is, and what it means

The profile at 13.9 ms is 2,827 kernels averaging 5 µs: the 290 projections' GEMV 3.6 ms, the fused norm 1.7, the routed-expert GEMMs 1.6, residual glue 1.4, the fp32 head 0.4, attention 0.4. The bytes the step actually touches (about 15.9 GB: 5.9 GB of fp8 projection weights, 4.5 GB of routed experts — 240 expert calls × 18.8 MB — 2.7 GB of bf16 attention output weights, 2.65 GB of head) would take ~2.5 ms at HBM rate. Corrected November 7: this first said about 9 GB and ~1.2 ms; the routed experts and the bf16 wo_a weights had been left out of the count. The step is latency-bound inside the graph: most kernels move under a megabyte and take 3–20 µs regardless. There are 3–4 ms of straightforward wins left on the 1-row path (in progress), and then the structure has to change: fewer, longer kernels (milestone 2), or more rows per step (milestone 1), which amortizes every latency-bound kernel except the MoE and the head. The design memo for the 6-row speculative step says plainly that no 6-row configuration of today's kernels beats 7.36 ms — the 5.0 ms bar needs the fused chain — and the 6-row measurement followed the same morning (section 8). The 1-row step itself reached 10.5 ms after this profile (T9b-4: the fused norm's reduction split, the split-K reduce folded into the GEMV, three pairs of same-input projections merged).

Parity against production, and what we got wrong

Against production's own greedy outputs over eight fixture prompts and 256 tokens, the runtime is token-identical on two, and on the other six it diverges at a near-tie (positions 1–152) with zero top-1 disagreements before the split and mean |Δlogprob| ≤ 0.05 nats on the shared prefix — unchanged at every rung above. We first suspected the KV-cache format (the reference stores the compressed latent through an FP4 quantize-dequantize; vLLM stores MXFP8); swapping ours to vLLM's moved the divergence points and did not shrink the drift, so that hypothesis is dead. The decisive check was production against itself: vLLM with one KV-format knob changed scores the same 2 of 8 against its own baseline, at the same token positions on three prompts. The charter's parity bar ("no worse than production's run-to-run spread") therefore measures vLLM against itself and nothing else can meet it; a replacement bar (bit-identical self-replay; ≤ 0.05 nats and zero weighted top-1 flips against production; tiered placement bit-exact against all-HBM) is proposed in PROJECT.md and awaits James's decision. Two other misses: the per-role nsys breakdown of production never came (both profiler-wrapped boots stalled once the capture range opened; parked), and the first GEMV round was numerically perfect and slow — no split-K, serial rows — and needed a second round. The glue census also undercounted: it counted launches correctly (~5,300) but at 1 µs each; measured they were 2.3 µs, so the glue was 12 ms, not 5.

The Station lane is dark between measurement windows. Production serves from the four-DGX-Spark SGLang lane. Everything above is in the public repo with its receipts: github.com/jmeadlock/gb300-dsv41-decode (Apache-2.0; docs/LEDGER.md is the running log, docs/NIGHT-LADDER-20261107.md the night's table). Runtime code was written by headless coding workers on claude-opus-5-5 from written handoffs, and measured, merged and fact-checked in one Milo session on claude-fable-5-1.

8. The headline instrument, and what the bar turned out to be

The number the charter asks for exists (November 7, morning). The runtime now runs DeepSeek's speculative decode the way production does: a 6-row verify step (the last accepted token plus five drafts), acceptance by exact greedy match with the confidence head off — what vLLM 0.31 does for this model — then a three-stage DSpark draft, all on the device inside two CUDA graphs. Both halves were proven exact first: the 6-row verify step is bit-equal to the reference model at 6 rows (684 fp32 logit rows, every cache tensor, both compressor parities), and the draft stage is bit-equal to the reference forward_spec (43 draft sets, all three rings). The loop's own accept/commit/emit work costs nothing measurable: one graph and two graphs give the same step time to 0.02 ms.

The headline instrument: ms per 6-row DSpark stepHorizontal bars on a linear scale: five headline runs on November 7 from 40.8 to 20.3 ms a step, production vLLM at 7.36, and the charter's original 5.0 bar. The headline instrument: ms per 6-row DSpark step (measured, cuda events, steps ≥ 32, 8 prompts) linear scale · each step = one 6-row verify graph + one 5-draft graph · 2.90 tokens accepted per step on this fixture (run 10) verify 6 rows + draft, as first wired05:19 · verify 23.0 + draft 17.740.8 msdraft stage on the fast kernels06:24 · verify 23.0 + draft 2.925.9 ms6-row MoE levers (run 3)06:49 · verify 19.9 + draft 2.922.9 msfixed-order fp32 (run 4)07:57 · verify 19.4 + draft 2.922.3 msafternoon round (run 10, section 8.1)15:45 · verify 17.4 + draft 2.9 · stream = 1-row step on 8/820.3 msproduction vLLM (Pg61)same model, same box, other instrument7.36 msthe original barcharter; restated: beat 7.365.0 ms 0 ms10 ms20 ms30 ms40 ms Same model, same box, same metric. Per token: ours 7.0 ms (run 10), production about 2.6 ms (7.36 ÷ ~2.8 accepted tokens a step).

Figure 6 — The project's own metric, day one. Receipts: docs/headline-20261107{,b,c,d,i-plan-gemv,j}.json, docs/HEADLINE-20261107.md, docs/LEDGER.md.

RunWhat changedms/stepverify + drafttokens/stepms/token
1As first wired: the 6-row verify with 16 of 18 fused glue clusters still on reference glue, the draft stage on reference Linear and glue.40.823.0 + 17.72.7714.7
2Draft stage ported to the same kernels as the verify (MXFP8 GEMV at 5 rows, DeepGEMM over the stage expert stacks, fused glue, FlashMLA): 17.7 → 2.9 ms.25.923.0 + 2.92.769.4
3The 6-row MoE: expert GEMM groups aligned to 16 rows instead of 128 (fastest of nine bit-equal layouts, 121 vs 155 µs a layer), staging only cold experts, a one-warp dedup kernel (106 → 6 µs), spare slots 8 → 16 so no step overflows.22.819.9 + 2.92.768.3
4Fixed-order fp32 kernels for the two cuBLAS sites (the mHC mixes and the head): bit-equal across 1 and 6 rows by construction, where cuBLAS picks a different algorithm per row count and differs on 86–91 percent of elements; also nearer the fp64 oracle (mean 2.0 ulp vs 10.0 on the head).22.319.4 + 2.92.977.5
5–10The afternoon round (section 8.1): adaptive expert residency, the head read as bf16, one reduction order per glue cluster at every row count, the router with its 6-row plan in one launch on a GEMV gate, and the ratio-2 compressor's projections on the fixed-order fp32 kernel. The stream now equals the 1-row step's on 8 of 8 prompts.20.317.4 + 2.92.907.0
—Production vLLM (Pg61), 6-row DSpark step, nsys instrument.7.36~2.8~2.6

Eight fixture prompts, 256 new tokens each, greedy; cuda events over live steps ≥ 32 of each sequence, pooled. Tokens per step by prompt class: prose 2.2, code 3.9, math 4.6, json 4.5, shell 3.7, tool 5.0, long 2.4, story 2.0 — production reports 2.6–2.9 on the same fixture. Every run replays byte-identically; the emitted token stream equals the 1-row greedy stream on 2 of 8 prompts and diverges at a near-tie token on the others (after run 4 the only place the 6-row step still differs from the 1-row step by an ulp is the routed-expert GEMM, which runs a different group shape per row count — the same class of difference production shows against itself; matching the two shapes is the next rung).

What the bar turned out to be

A 6-row step touches about 22 unique routed experts per layer (6 rows × 6 experts, deduplicated), 40 layers, 18.8 MB each: about 17 GB of expert weights, and about 29 GB in all once the dense projections, the head, and the draft stage are counted (the fusion design memo's shape arithmetic, modeled, not measured). At the copy rate we have measured on this box's HBM (6.3 TB/s) that is 4.6 ms of pure data movement, and at the best rate any kernel class has reached here, 6.3 ms. So 5.0 ms a step needs 92 percent of HBM bandwidth for the entire step, with under half a millisecond for everything that is latency-bound. Production's 7.36 needs 3.9 TB/s. The bar is a bytes bar, not a launch-count bar, and no fusion design meets it with stock weights at 6 rows. The two things that are reachable are beating 7.36 (the fusion ladder models about 9 ms, a persistent per-layer kernel about 7.3), and ms per token — the number a user feels, which the drafts turn step time into. James restated the bar the same morning: beat 7.36 and publish ms per token; 5.0 stays on record as the original, with this arithmetic as the reason. The three charter amendments the first day raised (the KV wording, adaptive residency, and the parity bar as a class — bit-identical self-replay always, and against production zero top-1 disagreements before the first near-tie split) are approved.

A comparability caveat, found November 7 while building the gap ledger. Production's 7.36 ms is not a one-GPU number. Its expert map keeps 255 experts a layer in GB300 HBM and the other 129 "cold"; the cold experts are not read over the chip-to-chip link — they are computed on a second GPU, an RTX PRO 6000 sidecar, and the GB300 pays about 0.02 ms a step for them. This project's charter is one GB300, and in our 6-row step the cold experts are copied in from Grace memory each layer at about 260 GB/s, which costs 4.55 ms a step — the single largest line in our step. The bar stays 7.36 as James stated it; what changes is how it should be read: we are trying to beat a two-GPU number on one GPU, and the honest like-for-like comparison (production with its sidecar off) is not production anymore. We will publish both framings whenever we publish a number against the bar.

8.1 The afternoon: five rounds in parallel, two that paid, two that taught

With the instrument in place, the afternoon ran five coding rounds at once — each in its own git worktree with its own files, each box-verified in turn — against the question the morning left open: why does the 6-row step's emitted stream match the 1-row step on only 2 of 8 prompts, when every kernel in the 1-row step is deterministic? The 6-row step is the same arithmetic; a difference can only come from a kernel that reduces in a different order when it sees six rows instead of one.

What did not pay, and what it proved. The first suspect was the expert GEMM: at one row the runtime calls DeepGEMM one way, at six rows another. Making both calls the same shape made every row's expert output provably bit-equal across row counts — and changed nothing: still 2 of 8, and the one-row step got 1.7 ms slower (36 padded blocks for one row). Kept as an option, off by default. The second was the GEMV: a round-2 research memo modeled that the 208 small projections paying ~8 µs each were latency-bound for want of bytes in flight, and that a rewrite would save 2 ms. The rewrite (wide tiles, several K-tiles in flight, the split-K reduced inside the kernel in a fixed order) is row-invariant and nearer the fp64 oracle, and saved 0.26 ms (3.41 → 3.15 ms of GEMV per 1-row step; receipt docs/gemv3-bench-20261107.json). The model was wrong: the fixed cost of a small kernel is ramp and tail, not memory latency, and a better GEMV cannot remove it. Fewer launches can. That is milestone 2's thesis, now with a measurement behind it. Branch parked, finding recorded.

What paid. Four rounds landed on main, each gated on the box:

Day's end: 20.3 ms a step, 2.90 tokens a step, 7.0 ms a token, from 22.3 / 7.5 at the morning's close; production 7.36 / ~2.6. The 6-row verify step is now, bit for bit, the 1-row step's arithmetic on every fixture prompt, and the parity bar against production (zero margin-weighted top-1 flips over the shared prefix) still holds on all eight. What the afternoon settled: the cost that is left is launches, not bytes.

8.2 The evening: five rungs of the fusion ladder, measured, none adopted

With the step bit-identical to its own 1-row arithmetic, the evening went at milestone 2's thesis: the remaining 13 ms is launches and per-kernel fixed cost, so cut launches and overlap kernels. Five rungs of the design memo's ladder were built in parallel, each in its own tree, each box-gated bit-identical before timing. Here is what the box said, in the order the memo ranked them by milliseconds per engineering day.

RungMemo's modelMeasured at 6 rowsWhat actually happened
L1 — the mHC mixes (which have a sublayer of slack) moved onto a forked branch of the captured graph, 160 launches off the critical path, bit-identical by construction−0.55 to −0.9 ms1-row step 10.11 → 9.73; 6-row step 20.30 → 20.65, with the fused variant and with a high-priority side stream alikeThe branch's fp32 kernels compete with the verify's GEMVs for SMs. Stream priority is not reserved capacity. A real 1-row win; a loss at the shape that matters.
L10 — the q/kv tail: pair GEMV plus two norm/RoPE/quant kernels in one launch (121 → 41 launches a step), bit-identical−0.21 msfirst form 21.80 (one CTA serialized six rows' tail); fixed form 20.45; per layer-call 17.9 → 21.4 µsThe GEMV loses about 9 µs when it carries the tail — fewer launches, more time. Wins only at 1 row (17.4 → 16.2 µs).
L5 — programmatic dependent launch (PDL) plus pre-wait L2 weight prefetch on every kernel we owna clean standalone probe (40 GEMV→glue pairs under graph capture) gave −1.3 µs per edge, −5.1 with prefetch, bit-identical; 569 owned-to-owned edges in the step → 0.7–2.0 msPDL on: 20.30 (no change). With prefetch: 20.63. 1-row 10.17 / 10.54.The probe's gain did not survive integration. And the GPU test caught a hazard the headline could not: with PDL on, the 6-row eager step's caches differ from the plain step — one consumer reads a producer's output before its wait. Logits and streams were untouched, which is exactly why a stream-only gate is not enough.
L3 — DeepGEMM's MegaMoE for the routed experts, the kernel production uses (81 µs a layer there)−1.6 to −2.6 msreal layer 0: 92 µs a layer against our 130–165 — fast, as advertised. Routed output vs ours: 82 percent of elements differ, max 8.5 absolute; distance to the fp64 oracle 0.0134 vs our 0.0017 (relative L2, 8×)Not a reduction order: block-scaled tensor-core math with the second GEMM's input quantized from fp32 instead of bf16. Also needs gate/up weights interleaved eight rows at a time, which our expert pool — and every reader of it — does not do; a copy is 137 GB; the in-place re-layout is a multi-day change to six files. Spiked, documented, not funded.
(morning) GEMV rewrite with many bytes in flight−2 ms (a research memo's latency model)−0.26 msThe fixed cost of a small kernel is not memory latency.

Receipts: docs/headline-mixes-fork-20261107.json, docs/headline-qkv2-20261107.json, docs/headline-pdl-{on,prefetch}-20261107.json, docs/dg-mega-spike-20261107.json, docs/pdl-gpu-test-20261107.log, docs/DAY2-REPORT-20261107.md. Same-config repeatability on this fixture is 0.02 ms, so every delta above is real.

Five for five against the thesis. Every rung that removed launches or overlapped kernels failed to convert at six rows; the two levers that paid today removed bytes (the head read in its checkpoint dtype) or fetches (expert residency), or fixed numerics. So the launch-count story is at best incomplete. A second-model review in the evening (GPT-6 Astra, briefed with every run and every negative, asked for its NEVER column) said the same thing more precisely: the brief conflated three costs — host dispatch, device scheduling and overlap, and execution that is genuinely memory- or dependency-limited — and "above the bytes floor" does not mean "the remainder is launches." Its one insisted change: before betting on a persistent kernel, build a matched-workload gap ledger against production — identical prefixes, positions, verify/draft shapes and residency conditions on both runtimes; verify, draft and step time; per-family attribution (routed MoE, shared expert, dense projections, attention, router, glue, unattributed); node and edge counts; measured overlap. We do not yet know how much of the 13 ms is the MoE kernel production uses, how much is scheduling, and how much is something neither runtime has been asked to show. That ledger is being built now; the next rung waits for it.

Our side of the ledger landed the same evening (docs/GAP-LEDGER-OURS-20261107.md; a 6-row step teacher-forced on production's own tokens, kernels attributed by family from a static profile, so diagnostic rather than additive). The sum of kernel durations is 19.25 ms of the 20.30 ms step: the step is very nearly a serial sum of its kernels, with about one millisecond of gaps in total. That is the arithmetic behind five rungs that bought nothing — there was only a millisecond of scheduling to win. The families: cold-expert staging over the chip-to-chip link 4.55 ms (40 layers × 114 µs, at the link's bandwidth); the routed expert GEMMs 4.45 (120 DeepGEMM calls); norms and the mHC mixes 2.50; the dense GEMVs 2.01; leftover torch elementwise 1.38; the head 1.02; shared expert 0.88; router 0.81; attention 0.41. So the next round goes at bytes and fetches, in that order: hide the cold staging under the hot experts' compute (copy engines, values untouched), keep more of the hot set resident, and the one kernel-speed lever with a measured number — DeepGEMM's MegaMoE at 92 µs a layer against our 130–165 — as a bounded numerics probe, approved, with no pool rewrite until its distance from the reference is understood.

Two corrections the reviewer made to this post's author, kept here on purpose: the "0.15 ms" attributed to the compressor kernel in section 8.1 is a net change between two runs whose accepted-token trajectories also differ, not an isolated cost (a frozen-prefix A/B/A would be); and the oracle-distance gate I applied to MegaMoE was my own operationalization, not the approved parity bar, which is defined against production — a bounded numerics probe is fair, a quiet gate swap is not.

8.3 The night of November 7: bytes and fetches, and a kernel we measured and declined

The gap ledger said the step was a serial sum of kernels and that its largest line was cold experts copied over the chip-to-chip link. The night went at that line, plus one rule change from James: the goal is better overall inference for everyone who uses the recipe, so every measured, bit-identical win counts — no size threshold. Four results, all box-measured, all bit-identical unless stated.

RungMeasuredNote
Expert residency on by default (the swap policy from the afternoon, re-measured on the current tree)20.30 → 19.93 ms; tokens and accepts equal to off0.7 swaps a step; cold share 0.94 → 0.65 percent. The cell measured second, with the policy warmed, ran 18.5 — the number a long session sees; reported, not the headline.
The small wins, un-parked under the new rule: the mixes fork and the fused q/kv tail, each a loss at six rows and a win at one, now row-conditional defaults1-row step 10.11 → 9.97 ms, byte-identical stream; 6-row unchangedThe GEMV rewrite we had parked (−0.26 ms of kernel time) was re-measured in the whole step: +0.16. Kernel time is not step time. It stays opt-in.
MegaMoE (DeepGEMM's routed-expert kernel, the one production uses), the approved bounded numerics probedrift source confirmed on real layers: the second GEMM's input quantized from fp32 — a torch model of the kernel matches it to four digits, and a one-line bf16-before-quant patch brings its distance from the fp64 oracle from 2–8× ours to 1.04–1.4×, at the same speed (80–110 µs a layer at six rows against our 130–165). Then the test that matters: substitute it on 4 of 40 layers, teacher-force production's 1,476 tokens, count top-1 flips against our own stream — stock 58, patched 49, max |Δ log-prob| 2.0 and 1.4For scale: our whole runtime differs from production by 55 flips on the same positions. Four layers of the patched kernel perturb the model as much as production's entire kernel set. Extrapolated to forty, far outside the parity class we accepted. Declined; 0.8–2.2 ms forgone, receipts kept, and no expert-pool rewrite.
Cold-expert staging overlapped with the hot experts' compute — the copy on a forked branch of the captured graph, joined per projection right before the GEMM that needs it, the shared expert hoisted into the gapfirst form 19.67; with per-projection launch shapes (a fast copy for the first projection, low-contention copies for the other two) 19.34 ms a step, 6.66 ms a token, streams exact 8/8, replay 8/8Copy engines measured 366 GB/s over the link versus the kernel's 263, but a memcpy node cannot follow which experts are cold each step inside a captured graph, so the kernel form won. The first box run's mini-fixture test failed; the cause was the test's own stale KV cache, not a race — kept in the log. A diagnostic run that skips staging entirely (wrong outputs, so not a result) ran 16.64: staging still exposes about 2.7 ms, and that is the next rung.

Receipts: docs/headline-resid2-20261107.json, docs/fl-sw-gemv{,3}-20261108.log, docs/m3n1-downstream{,-bf16q}-20261107.json, docs/m3n1-layers{,-bf16q}-20261107.json, docs/headline-s1-{48,shapes,nonediag}-20261107.json; docs/M2-L3-NUMERICS.md for the MegaMoE method.

Later the same night: what the staging kernel was actually moving. With the copy hidden, 2.7 ms of it was still exposed, so the next worker counted bytes instead of guessing. Each layer-call the stage kernel moved 6–21 MB of cold expert weights — and about 46 MB of the hot experts' scale factors, read and re-written every layer, every step. The reason is a fallback that every headline run had been logging: the pre-packed scale-factor pool needs 13 GB of HBM and 5 GB was free, so the runtime silently took the per-step path. Two fixes, both bit-identical, both box-measured against an interleaved baseline. First, pack as many layers' scale factors as fit (17 of 40, after trading four of the eight residency slots for the room): 19.06 ms. A tempting third arm — halve the staging slots to fit 24 layers — ran 18.92 and was thrown out: it overflowed the slots on 13 steps and broke replay; the cold-expert histogram (at most 14 cold experts in a layer-call, more than 8 on 0.25 percent of calls) says sixteen slots stay. Second, the real fix: the scale factors cannot sit physically beside their weights (DeepGEMM addresses the two through separate TMA maps with implicit strides), but they can live slot-parallel in the pool, indexed by the same group index — so no hot scale factor is ever copied, there is no scale-factor pool at all, and 5.2 GB of HBM comes back. 18.74 ms a step, 6.46 ms a token, streams exact 8 of 8, replay 8 of 8, the 1-row step 9.98 → 9.76 with a byte-identical stream. Receipts: docs/headline-s2-{A,C,D}-20261108.json, docs/stage-hist-20261108.json, docs/headline-s3-{A,B}-20261108.json, docs/M3-S2-STAGING-BYTES.md, docs/M3-S3-POOL-SF-LAYOUT.md.

The night's ladder, then: 20.30 → 19.93 → 19.34 → 19.06 → 18.74 ms a step; 6.99 → 6.46 ms a token; every step bit-identical to the 1-row arithmetic. Still exposed: about two milliseconds of cold-expert bytes over the link, which the freed HBM can now be spent against.

Where that leaves the ladder: 18.7 ms a step and 6.5 ms a token on one GB300, bit-identical across runs and to the 1-row arithmetic, against production's 7.36 (two GPUs; section 5). The day's lesson, twice over: a kernel being fast is a fact about the kernel; whether the step gets faster — and whether the model stays the model — are separate measurements, and we take both before anything becomes a default.

8.4 Sunday morning: production on the same prompts, and two columns from now on

Three decisions from James at breakfast, each measured by mid-morning.

Production, matched. Until now the production number on this page came from a different prompt. With permission, production's own container was booted on the Station in its real configuration (255 hot experts a layer in HBM, the cold ones computed on the RTX PRO 6000 sidecar), driven through the same eight fixture prompts, and shut down again — twenty minutes of box time. Its outputs matched its own parity record on all eight. On the fixture: 7.66 ms a step, 2.97 tokens a step, 2.55 ms a token of decode wall time (2.70 including prefill). The unmatched figure had been 7.74, so nothing was hiding in the workload; the gap is what it looked like. One thing the matched ledger did surface about us: our wall time per token runs above our GPU time per token. Correction, same morning: this post first attributed that to host work between graph replays. Measured per prompt, the decode wall time equals the issued steps times the GPU step time — the loop is tight, there is no per-step host gap. The real costs are two others. First, the loop issues 9–15 steps after a sequence has finished before its poll notices (97 steps across the eight prompts, about 1.8 s), a fixed 0.2–0.3 s per request that dominates short answers. Second, and larger: prefill takes 3.5–4.2 s for a 31–54-token prompt, because it still runs the reference model's eager forward — a time to first token of about four seconds against production's tens of milliseconds. For anyone actually using this runtime those two outrank every rung on the decode ladder, and they are the next round (fixed by noon: section 8.5). Receipt: docs/pg61-drive-matched-20261108.json, docs/GAP-LEDGER-20261108.md.

Two columns. With the residency policy on, the step gets faster as a session runs: the policy learns which experts this conversation keeps asking for and promotes them, so the second time the same cell is measured it is a different machine. The headline instrument is deliberately cold — eight prompts, 256 tokens each, policy fresh — and it stays the headline, because every number on this page is comparable to it. From today the instrument also re-runs the same cell with the policy carried over and reports it as a steady-state column, what a long session would see. On the current tree: headline 18.62 ms a step, 6.41 ms a token; steady state 17.92 ms a step, 6.17 ms a token, cold-expert share 4.8 → 1.6 percent, the emitted tokens identical in both passes, streams exact 8 of 8. The columns are defined in docs/HEADLINE-COLUMNS.md; when this page quotes one number it is the headline.

Spending the memory. The scale-factor layout from the night returned 5.2 GB of HBM. Two ways to spend it on the cold-expert bytes still exposed were measured against each other. More residency slots (8, then 12, instead of 4): no change on the headline cell (18.77, 18.74), a small gain on the warm cell only. More hot experts per layer, from a map rebuilt from the same decode counts as the original (self-test: it regenerates the 295 map byte for byte): 301 per layer 18.62 with 3.9 GB still free; 304 per layer 18.56 with 1.7 GB free — under the headroom a longer context needs, so 301 is the default and 304 an option for short contexts. Both bit-identical, replay 8 of 8. Receipts: docs/headline-rs{24,28}-20261108.json, docs/headline-hot{301,304}-20261108.json, docs/HOT-MAP.md.

Ladder since the instrument went live Saturday morning: 40.8 → 22.3 → 20.3 → 18.74 → 18.62 ms a step (steady state 17.92); 7.0 → 6.41 ms a token (6.17). Production on the same prompts: 7.66 and 2.55, on two GPUs.

8.5 Sunday mid-morning: the two wall-clock costs, and a recipe you can run

The correction above named two costs no kernel profile shows. Both were measured, then fixed, before noon.

Overrun. The async loop issued steps ahead of the completion check, so every sequence ran 9–15 steps past its last token — 97 wasted steps across the fixture, about 1.2 ms per token. A probe put a number on the alternative: polling completion every step costs 13 µs of bubble, so the break-even against one wasted 18 ms step is 1,375 steps. Poll every step, then. Result: 8 overrun steps over the fixture (at most one per prompt), and decode wall time per token of 6.11 ms against 6.09 ms of GPU time — a host gap of 0.02 ms per token. Wall now equals GPU. Receipt: docs/headline-20261108-main-ov2.json, docs/poll-probe-20261108.log.

The four-second first token. The anatomy run said where it went: a 38–54-token prompt makes 3,300–3,700 calls into the reference's per-expert loop (about 90 distinct experts per layer, 40 layers), each about a millisecond of host time for 26 kernel launches — roughly 95,000 launches, and the GPU busy 19 % of the time. Syncs were not it (one sync per layer saved a tenth); launches were. The fix keeps every reference kernel and the reference's reduction order and strips everything around them: the per-expert work is grouped into the GEMM launches alone, spread over four CUDA streams, with the fill-with-zeros passes dropped (measured bit-equal). Prefill 4.35 s → 0.94 s averaged over the eight prompts; ~90,000 launches → 10,400; the first token, the logits, every cache tensor and the full 256-token streams bit-equal to the reference prefill on 8 of 8. Time to first token is now 0.82–0.95 s (0.82 s in the headline run at the shipped tree, 0.95 s measured under the profiler), against production's tens of milliseconds. What remains is 10,400 expert GEMMs at about 90 µs each: a true grouped kernel would model to 0.1–0.2 s, but it would be a different kernel from the reference's — a parity-class change, which this project does not make without a decision. Receipt: docs/prefill-anatomy-20261108.log, docs/prefill-bench-m4p2-20261108.json, docs/headline-m4p2-20261108.json.

Where that leaves one GB300, single stream, Sunday noon: first token 0.8–0.95 s; decode 18.62 ms a step, 6.41 ms a token (17.92 / 6.17 in steady state), about 160 tokens a second on the wall clock with no host gap; every stream byte-identical run to run and to the runtime's own 1-row greedy, the prefill bit-equal to the reference's, the decode kernels at the parity class stated in section 7 (55 flips against production over 1,476 teacher-forced tokens). Production on the same prompts: ~30 ms to first token and 392 tokens a second, on two GPUs.

Recipe v1. Everything on main is box-measured and bit-identical, so it ships now rather than at a round number — every bit matters if people run it. The recipe (image, launch, defaults, receipts, fidelity evidence, and what is not measured: concurrency, long context past 4k) is in J-M-Recipes: deepseek-v4.1-flash-standalone-decode-runtime. Deltas will land there as they are measured.

Where the 20.1 ms goes, and the next rungs

The 6-row profile is the 1-row profile plus the MoE: at 6 rows the expert GEMMs, staging, dedup and scale packing grew from about 4 ms to 12.5 before run 3's levers; everything on the attention, projection and norm side is row-invariant in time, as the design memo predicted. Next: the gap ledger against production (section 8.2), then whichever family it names; the fusion ladder's remaining rungs are measured one at a time, and the 1-row-only wins (the mixes fork, the q/kv tail) are kept for the parity instrument where they are wins.

Corrections

Section 7's 1-row bytes figure first read about 9 GB (~1.2 ms at HBM rate); the routed experts (4.5 GB) and the bf16 attention output weights (2.7 GB) had been left out. It now reads 15.9 GB (~2.5 ms) and is corrected in place. The 6-row design memo's first estimate for a fully ported 6-row verify step was 13–15 ms; measured, it is 19.9 inside the loop, and the gap is the MoE at 6 rows, which the memo had costed at the 1-row figure.