One byte in 250 calls: a 6% faster DeepSeek-V4.1-Flash recipe, and the rounding bug behind it
Pg62 moves 38 more attention layers of DeepSeek-V4.1-Flash onto Garner McCloud's fast decode kernel and rebuilds that kernel to round exactly like the engine's stock op: +6.1% for one user. Finding out why it was not exact turned up the same rounding bug in the kernel we had been serving since October: one output byte in about one call in 250.
1. What changed
The lane is one NVIDIA DGX Station GB300 with the RTX PRO 6000 Max-Q in the same chassis as an expert sidecar, serving DeepSeek-V4.1-Flash-1017: 510 GB of weights as shipped, an fp8 KV cache, the full 1,048,576-token context, 32 seats, a KV pool of 11,696,582 tokens, vLLM v0.31.0. The engine is Garner McCloud's DS4.1 engine (MIT, pinned at commit 8043a681) under our policy and patch set. W1 is his family of decode-attention kernels. Pg61 has served since October 29, 2026. Pg62 is Pg61 plus two changes to decode attention and nothing else.
An FP8 fast path. Pg61 ran W1 only on the window-only layers: target layers 0 and 1, and the drafter's layers 40, 41 and 42. The compressed target layers 2 to 39 keep their KV cache as 528-byte fp8 records and ran the stock FlashMLA op. Pg62 routes those 38 layers to W1 whenever the decode step carries 1 to 16 rows, and to the stock op at 17 rows or more. Rows are the requests in the batch times the tokens being checked, so a single user is 6 or 8 rows a step and never reaches the ceiling. The routing itself was first written by an earlier agent session (GPT-6 "Astra") under James; this round made it exact and measured it.
Exact rounding. Both W1 extensions are rebuilt to round the way the stock op does, behind a compile flag: the softmax sink term computed the stock way, plus three smaller forms matched to stock. Section 2 explains why that was necessary, and why it costs nothing.
Figure 1. Which code runs decode attention on each layer. Pg61 ran W1 on the 5 window-only layers. Pg62 also runs it on the 38 compressed layers whenever a step carries 1 to 16 rows; wider steps stay on the stock op.
2. The rounding difference
The attention softmax carries a "sink" term, sink*log2e - m. W1 computed it as one fused multiply-add: multiply and subtract in a single instruction, rounded once. The stock op multiplies, rounds, subtracts, rounds again. Both are correct to within one float step (1 ulp) and are almost always identical. Ours differed because the value is quantized to an FP8 code immediately afterwards.
Each FP8 code covers a wide span, and almost every value sits well inside one, where a nudge of one float step changes nothing. Only a value within one step of the midpoint between two codes lets that nudge pick the code. In the call we captured and rebuilt offline, the stock op wrote FP8 code 110 and W1 wrote 111: the value sat one float step above the 110/111 midpoint. That was one data byte in the whole call's output; no scale byte differed.
Figure 2. The captured call. Both paths are legitimate ways to compute the sink term; they differ by one float step, and this value happened to sit one step from the boundary between two FP8 codes. Splitting the fused instruction is what fixes it: on that fixture, unfusing the sink term alone gave 110, while the exact-divide switch alone and each no-flush-to-zero switch alone still gave 111.
Greedy decoding turns one byte into different text. The model takes the highest-scoring token at every step, so a one-byte change nudges the scores, and where two candidates were nearly tied the other one wins and the texts part ways. On our 8 fixed prompts (greedy, up to 256 tokens, text and top-20 log-probabilities compared token for token against the Pg61 configuration with the new path off) the reference matched itself 8 of 8. The new path before the fix scored 5 of 8, with the code, prose and shell prompts differing. After the fix: 8 of 8, and 8 of 8 again on the live lane after the cutover.
3. How we proved it
The instrument is a shadow compare. A test build, never the one served, runs both W1 and the stock FlashMLA op on every decode-attention call inside the CUDA graphs, on the same inputs, and compares the two output buffers byte for byte on the device. It counts the calls with any differing byte. The model always consumes the stock result, so one differing call cannot snowball into the next. The workload was a 16-prompt prose pass, the 8-prompt greedy check, both single-user speed sets, one 8-user cell, and long-context runs with prompts up to 571,462 tokens.
Figure 3. The shadow compare and its four headline counts. Both kernels run on every call inside the graph, the bytes are compared on the device, and the model carries on with the stock result.
| what was compared | calls compared | calls with a differing byte |
|---|---|---|
| the new FP8 path as first written, before the fix | 19,152 | 3 |
| Pg61's shipped window-only kernel, serving since October 29 | 17,797 | 70 |
| Pg62, FP8 path | 132,240 | 0 |
| Pg62, window-only path | 17,779 | 0 |
| Pg62, prompts up to 571,462 tokens, FP8 path | 46,702 | 0 |
| Pg62, prompts up to 571,462 tokens, window-only path | 6,913 | 0 |
The 70 differ by layer as follows: target layer 0, 1 of 3,569; layer 1, 17 of 3,569; drafter layer 40, 7 of 3,553; layer 41, 15 of 3,553; layer 42, 30 of 3,553. Every one was a single data byte. A window-only rebuild with only the sink term changed read 0 of 17,733 (and 0 of 132,354 on the FP8 path), so the sink term accounts for every difference we saw.
One dead end is worth recording. Two replays of 76 captured real inputs each passed 76 of 76 while the served lane kept failing 5 of 8. A difference that hits one call in 6,400 and depends on the exact values does not show up in a replay of a few steps, which is why counting every call was the only instrument that worked. Three other candidates left the oracle at 5 of 8: the exact divide alone, a different kernel plan, and removing the early launch trigger.
4. What it bought
One window, one harness, with stock Pg61 controls booted before and after the Pg62 binaries. The harness runs 20 prompts of 256 tokens per set at greedy decoding, 3 measured reps after warm-ups.
Figure 4. Pg61 (two control boots) against the Pg62 binaries in the same window. The printed values are ranges, not means; the 32-user control cells spread 13% in this window.
| cell | Pg61 (two control boots) | Pg62 binaries |
|---|---|---|
| 1 user, prompt set A, decode tok/s (3 reps) | 413.88–414.72 (mean 414.22) | 439.36–439.57 (mean 439.47) |
| 1 user, prompt set B | 409.88–410.80 (mean 410.25) | 435.28–435.42 (mean 435.33) |
| 1 user, ms per step, set A | 7.639–7.654 | 7.205–7.209 |
| tokens per step, A / B | 3.125 / 3.091 | 3.123 / 3.089 |
| 8 users, aggregate tok/s | 1,085.1–1,108.6 | 1,087.6–1,094.9 |
| 32 users, aggregate tok/s | 1,626.1–1,833.2 | 1,794.1–1,809.9 |
| first token at a 60,953-token prompt | 1.462–1.466 s | 1.458 s |
| KV pool, tokens | 11,696,582 | 11,696,582 |
Both single-user sets gain the same amount: 439.47 / 414.22 = 1.061, and 435.33 / 410.25 = 1.061, so +6.1%. The 8-user, 32-user and first-token cells did not move. Tokens per step shift slightly (3.125 to 3.123) because the model now follows stock values rather than Pg61's on these prompt sets, which is the visible sign that the fix changes output on some prompts.
The reason only one user gains is the 16-row ceiling: at 8 users a step is 28 to 32 rows and at 32 users 128, past the limit, so those steps still run the stock op. Extending the FP8 path to 17 to 76 rows on the cluster kernel was byte-exact in 152,342 calls but did not move 8- or 32-user throughput, so it is not in the recipe. On the live lane after the cutover, the same harness read 439.33 tok/s at 7.214 ms a step.
Two more views, as context. Per class at temperature 1.0 on the live lane, on November 15, 2026 (40 prompts of 256 tokens per class, mean of 3 seeds): prose 405.3, structured 441.1, code 556.2, shell 561.9, tool-JSON 489.6, for a class mean of 490.8 tok/s. Pg61 read 450.6 on October 29, a different day and a different boot, so that comparison is context, not a paired ratio; the paired number is the +6.1%. From a caller's seat the same day: 8 streams at 1,094 tok/s aggregate, a 40-token turn in 85 ms, first token idle 32 ms, a quiet step of 7.0 ms, and a single 400-token stream at 330 tok/s median of 3 (305 / 330 / 408, a noisy cell; Pg61 read 343 on October 30).
5. Quality check
A quality suite ran against the live Pg62 lane on November 15, 2026, with the pass bars written before the first request. The reference is Pg61's stored per-item results from October 29, paired by item: a cross-day comparison, not a same-window control, since Pg61 cannot be re-run without ending the soak.
| bar | Pg62 | Pg61 reference | paired reading |
|---|---|---|---|
| needle-checksum, 500 requests | 500/500 exact | 500/500 | pass |
| GSM8K, 1,319 items | 1,284 (97.35%) | 1,279 (96.97%) | 5 lost, 10 gained, p = 0.30 |
| BFCL held-out, 1,311 items | 1,024 (78.11%) | 1,026 (78.26%) | 17 lost, 15 gained, −0.15 pt against a −1.0 bar, p = 0.86 |
| MMLU-Pro, 1,000 items | 871 | 877 | 19 lost, 13 gained, p = 0.38 |
| tool-eval, hard mode, no bar | 93 | 89 and 91 | reported, not scored |
No measured loss, and not a measured gain either: none of the three paired differences stands out from item-level noise at these sample sizes. Six of six bars passed, the sixth being a clean watchdog after the suite.
6. What is not done
- Experimental. The 48-hour soak ends Monday, November 16, 2026 at 11:11 PM CST. No soak has completed on this lane yet.
- Not run on Pg62: a temperature-above-zero quality check, a teacher-forced comparison, and the tool gauntlet.
- Rollback to Pg61 is one script that has only been dry-run.
- "Exact" is a narrow claim: W1 decode attention against the stock FlashMLA op, on the tested workload. It says nothing about the rest of the engine against bare vLLM; the cluster kernel and FP4 records are untested on this lane; and no call at 1 to 5 rows was compared.
- Pg61 was never byte-equal to the stock op on the window-only layers, and Pg62 is, on the tested workload. So Pg62's greedy output differs from Pg61's on some prompts by design. Anything that compared text against Pg61 as a reference will see that difference.
- Upstream. The rounding difference is in Garner McCloud's W1 source; the fix is a small patch on it, published in the recipe. We are sending him the patch.
- The published launcher rebuilt both kernel binaries byte-identically on the reference box, but no lane has been booted from a launcher-built tree; the serving tree was assembled by hand.
7. Reproduce it
The recipe holds the launcher, the W1 patches and every receipt behind the numbers above: J-M-Recipes, deepseek-v4.1-flash-vllm-uva-dspark-sidecar, with the round bundles results/2026-11-14-pg62-w1-exact/ and results/2026-11-15-pg62-t2-t3/. The W1 changes are three patches plus two build drivers under patches/garner-8043a681/w1-exact/, all default-off, so a build without the flag behaves exactly as before.
Earlier on this lane: the sidecar recipe itself and what the second GPU is worth.
Credits
Engine and the W1 kernel family: Garner McCloud, dgx-station-recipes at commit 8043a681 (MIT). The rounding fix is a small patch on his source, and we are sending it to him. The FP8 routing was first written by an earlier agent session (GPT-6 "Astra") under James Meadlock. The diagnosis, the fix, the measurements and this write-up: Milo, with Claude Opus 5.5 and Sonnet 5.5 as desk workers on the kernel analysis.