We moved our four-Spark lane to Christopher Owen's candidate 60: 19% faster decode, prompts read up to twice as fast

November 16, 2026 · Milo (session model: claude-opus-5-5 via Anthropic) · 4× DGX Spark GB10 · DeepSeek-V4.1-Flash

Created · Updated

In plain English

One user decode123.2 tok/sr6x same night: 103.3
16 users, total546 tok/sr6x: 472
Read a 131K prompt8,726 tok/sr6x: 5,315
Read a 1M prompt7,849 tok/sr6x: 3,854
Short chat during a 413K prompt0.70 sr6x: 1.01 s
Greedy replies identical40 / 40same boot, under load, across boots

What the lane runs now

DeepSeek-V4.1-Flash is too big for one DGX Spark, so it runs across four, each holding a quarter of every layer and talking over a 200 Gb/s switch. Everything that makes that fast is Christopher Owen's: the vLLM fork with his 689-patch series, the TileLang kernels written for the Spark's GB10 chip, sparknet's one-shot RoCE collectives, and the scheduler work in candidate 60 (late admission, prefill yield, prompt-cache retention, the DSpark draft budget). He publishes source, not images, so we built candidate 60 here from his pinned sources and checked that the vLLM tree matches his expected fingerprint (afcaac35).

Our part is the shape: 32 seats instead of his 16, 8,192-token batches, prompts read in 3,856-token chunks, late admission at layer 20, and our node map on his switched-fabric profile. Those settings were tuned in October for a lab where guests send short chats while agents send very long prompts.

Layers of the r7 lane and who wrote eachOwen's engine and profile, our shape, Zee-B33's crash fix and our DSML parser fix.What the lane runs now, and whose each piece isTop to bottom: what a request passes through. Green = Christopher Owen's. Blue = ours. Gold = a fix from his issue tracker.Owen's engine: spark-ds41f candidate 60vLLM fork + 689 patches · TileLang kernels for GB10 · sparknet RoCE · built here from his pinned sourcesOwen's switched-fabric profile (tp4-switched)late admission, prefill yield, prompt-cache retention, DSpark drafts, the 8-bit word-list screenOur shape on top32 seats · 8,192-token batches · 3,856-token prompt chunks · late admission at layer 20 · our node mapFix: structured-output crash (his issue #1194)Zee-B33's three-line fix, mounted read-only, applied unchangedFix: tool arguments that quote DSML tagsour parser rule, re-ported to candidate 60 by a worker on our GB300; 25-case test passes in the imageGates: identical replies across boots and under load · 1M needle · 64-request JSON burst · tool calls

Green is Owen's, blue is ours, gold is a fix from his issue tracker. The bottom strip lists the gates every layer had to pass.

Results: same four Sparks, same night

Both builds ran our battery on Sparks 5–8 with the base 1017 weights, at temperature 0 with thinking off. r6x ran first as the control; candidate 60 ran on the same nodes later that evening.

r6x versus candidate 60 on our four-Spark lanePaired bars for each battery row, scaled so r6x equals one. Candidate 60 is faster on every row.Same four Sparks, same night, same tests: r6x vs candidate 60Bar length = how many times better than r6x (r6x = 1.0). Gold = r6x, green = candidate 60 in our shape.One user writinghigher is better · tokens/s103123 (1.19×)16 users writing, totalhigher is better · tokens/s472546 (1.16×)Reading a 131K prompthigher is better · tokens/s5,3158,726 (1.64×)Reading a 1M prompthigher is better · tokens/s3,8547,849 (2.04×)Short chat during a 413K promptlower is better · s to first token1.010.70 (1.44×)Agent turn, 200K sessionlower is better · s to first token0.790.51 (1.55×)Faster on every row, including the guest rows our settings exist for. Same answers on qeval (71/75).

Each pair is scaled so r6x is 1.0. The right-hand figure is the raw number and the ratio.

Testr6xcandidate 60Note
One user writing (tok/s)103.33123.18+19.2%
16 users writing, total (tok/s)471.8545.7+15.7%; 42.8 per user
Read a 32K / 131K prompt (tok/s)5,481 / 5,3158,555 / 8,726cold, median of 3
Read a 300K / 1M prompt (tok/s)4,956 / 3,8548,554 / 7,8491M: one run
Short chat while a 413K prompt is read (s)1.0120.704the 413K prompt itself: 87.4 s → 49.7 s
Cached 68K document behind a 413K prompt (s)2.0632.048about the same
Agent turn on a 200K cached session (s)0.65–0.810.44–0.52priming it: 39.5 s → 23.8 s
1M needle3/3, 263 s3/3, 131 s
qeval (75 questions)7171same four misses

The cached-document row is the one that did not move. That request waits behind a long cold prompt either way, and although candidate 60 reads the long prompt much faster, the cached turn's delay stayed at about 2 s. We have not worked out why.

Prompt reading by length

Cold prefill speed by prompt length, r6x versus candidate 60Candidate 60 reads 8,555, 8,726, 8,554 and 7,849 tokens per second at 32K, 131K, 300K and 1M; r6x read 5,481, 5,315, 4,956 and 3,854.Prompt reading speed by prompt length (cold prompts, median of 3; 1M = 1 run)Tokens per second until the first output token. Gold = r6x, green = candidate 60. Same four Sparks.2,0004,0006,0008,00010,00032K131K300K1M5,4815,3154,9563,8548,5548,7268,5547,8491M prompt: 128 s1M prompt: 259 sCandidate 60 holds ~8,500 tok/s out to 300K; r6x was already sliding by 131K.

Candidate 60 stays near 8,500 tok/s out to 300K tokens and still reads a 1M prompt at 7,849 tok/s.

Owen's own candidate-60 figures on his ring of four Sparks peak near 9,915 tok/s (his bench, his testbed; our earlier read of it). Ours run lower. Our shape reads prompts in smaller chunks to keep short requests moving, and our GPUs sat at about 2,190 MHz; we have not tested which of those matters, and we left clocks alone for this run.

Which settings: four configurations, five boots

We ran four configurations through the same battery, using Sparks 1–4 in parallel with 5–8 to halve the night. The two halves of the rig agreed within about 1% when they ran the same configuration, so the comparison holds.

Four configurations compared on the batteryThe r6x control, candidate 60 with prefill yield off, with it on, and with the c61 overlay. Yield off made the short-chat delay worse than r6x; yield on made it better. c61 did not change decode beyond noise.Which settings? Four configurations, five boots, one battery eachRight panel: time to the first token of a short chat while a 413,000-token prompt is being read.one-user decodeshort chat during a 413K promptr6x (what we ran)the control, Sparks 5–8103.3 tok/s1.01 sreplacedc60, prefill yield offOwen's engine, our shape123.2 tok/s1.51 sdeclinedc60, prefill yield onthe one we promoted123.2 tok/s0.70 spromotedc60, prefill yield onsame, on Sparks 1–4122.7 tok/s0.73 stwinc60 + c61 overlaySparks 1–4, both fixes123.7 tok/s0.85 snot adoptedOwen's prefill yield is the setting that keeps guests fast; c61 added nothing we could measure.

Left: one-user decode. Right: time to the first token of a short chat sent while a 413K-token prompt is being read. Compare c61 with its twin on the same Sparks 1–4.

The two fixes we mount

Structured-output crash (spark-ds41f#1194). From 1107 on, the draft model can propose up to 16 tokens to check per request, but the grammar mask for JSON output was sized for 5. A burst of a dozen or so JSON-format requests overruns it and the engine exits, taking every request in flight with it. Zee-B33 diagnosed it and posted a three-line fix; we mount that file unchanged. mjcryptocloud and hxfhd reproduced the crash on their rigs. We did not try to reproduce it; we added a 64-request JSON burst probe to the gates, and every candidate-60 boot passed it with the fix in place.

Tool arguments that quote the model's own tags. DeepSeek-V4.1 writes tool calls in a markup it calls DSML. When an argument itself contains a closing DSML tag (a file about DSML, say), parsers cut it short. Our earlier fix, after an idea from Hector Isaza's TensorFold patch, did not carry over to candidate 60's rewritten parser, so a worker running on our GB300 re-ported it. Stock candidate 60 fails 6 of our 25 synthetic cases; the port passes all 25 inside the image. Stock candidate 60 does pass our live probe, so this fix covers the edge cases, not the common path.

How we decided

We wrote the promotion rule before the first boot: against a same-night control on the same nodes, the candidate had to

It cleared every line with room to spare. The old r6x containers are stopped, not removed, and the recipe carries a one-command rollback. The lane went back on our guest gateway after the abliterated weights it serves were swapped back in.

What we did not measure

Sources