We moved our four-Spark lane to Christopher Owen's candidate 60: 19% faster decode, prompts read up to twice as fast
In plain English
- This lane runs Christopher Owen's engine. Our four-Spark DeepSeek-V4.1-Flash lane is spark-ds41f, his vLLM fork, GPU kernels and network code. Tonight we moved it from his October build (r6x) to his newest, candidate 60, and kept our own settings on top.
- Faster on every row we test. On the same four Sparks, the same night: one user gets 123.2 tokens per second instead of 103.3 (+19%). A 1M-token prompt is read in about two minutes instead of four. A short chat that arrives while someone's huge prompt is being read gets its first token in 0.70 s instead of 1.01 s.
- Same answers. Our 75-question check scores 71 on both, with the same four misses. Greedy replies to 40 fixed prompts came out identical twice in a row, again under load from 15 other users, and again after a reboot onto different hardware.
- One of Owen's settings made the difference. His "prefill yield" lets short requests slip in while a long prompt is read. With it off, candidate 60 was worse than r6x for that short chat (1.51 s). With it on, better.
- What we added: two small fixes mounted on top. One is a three-line fix for a crash under bursts of JSON-output requests, written by Zee-B33 on Owen's issue tracker. The other is our own fix for tool calls whose arguments quote the model's own tool-call tags.
- What we skipped: his next branch, c61, aimed at one-user speed. On our tests it moved one-user decode by under 1%, inside the noise, and the 16-user total went down a little.
- The recipe is published: J-M-Recipes, four-Spark vLLM lane, with every receipt.
What the lane runs now
DeepSeek-V4.1-Flash is too big for one DGX Spark, so it runs across four, each holding a quarter of every layer and talking over a 200 Gb/s switch. Everything that makes that fast is Christopher Owen's: the vLLM fork with his 689-patch series, the TileLang kernels written for the Spark's GB10 chip, sparknet's one-shot RoCE collectives, and the scheduler work in candidate 60 (late admission, prefill yield, prompt-cache retention, the DSpark draft budget). He publishes source, not images, so we built candidate 60 here from his pinned sources and checked that the vLLM tree matches his expected fingerprint (afcaac35).
Our part is the shape: 32 seats instead of his 16, 8,192-token batches, prompts read in 3,856-token chunks, late admission at layer 20, and our node map on his switched-fabric profile. Those settings were tuned in October for a lab where guests send short chats while agents send very long prompts.
Green is Owen's, blue is ours, gold is a fix from his issue tracker. The bottom strip lists the gates every layer had to pass.
Results: same four Sparks, same night
Both builds ran our battery on Sparks 5–8 with the base 1017 weights, at temperature 0 with thinking off. r6x ran first as the control; candidate 60 ran on the same nodes later that evening.
Each pair is scaled so r6x is 1.0. The right-hand figure is the raw number and the ratio.
| Test | r6x | candidate 60 | Note |
|---|---|---|---|
| One user writing (tok/s) | 103.33 | 123.18 | +19.2% |
| 16 users writing, total (tok/s) | 471.8 | 545.7 | +15.7%; 42.8 per user |
| Read a 32K / 131K prompt (tok/s) | 5,481 / 5,315 | 8,555 / 8,726 | cold, median of 3 |
| Read a 300K / 1M prompt (tok/s) | 4,956 / 3,854 | 8,554 / 7,849 | 1M: one run |
| Short chat while a 413K prompt is read (s) | 1.012 | 0.704 | the 413K prompt itself: 87.4 s → 49.7 s |
| Cached 68K document behind a 413K prompt (s) | 2.063 | 2.048 | about the same |
| Agent turn on a 200K cached session (s) | 0.65–0.81 | 0.44–0.52 | priming it: 39.5 s → 23.8 s |
| 1M needle | 3/3, 263 s | 3/3, 131 s | |
| qeval (75 questions) | 71 | 71 | same four misses |
The cached-document row is the one that did not move. That request waits behind a long cold prompt either way, and although candidate 60 reads the long prompt much faster, the cached turn's delay stayed at about 2 s. We have not worked out why.
Prompt reading by length
Candidate 60 stays near 8,500 tok/s out to 300K tokens and still reads a 1M prompt at 7,849 tok/s.
Owen's own candidate-60 figures on his ring of four Sparks peak near 9,915 tok/s (his bench, his testbed; our earlier read of it). Ours run lower. Our shape reads prompts in smaller chunks to keep short requests moving, and our GPUs sat at about 2,190 MHz; we have not tested which of those matters, and we left clocks alone for this run.
Which settings: four configurations, five boots
We ran four configurations through the same battery, using Sparks 1–4 in parallel with 5–8 to halve the night. The two halves of the rig agreed within about 1% when they ran the same configuration, so the comparison holds.
Left: one-user decode. Right: time to the first token of a short chat sent while a 413K-token prompt is being read. Compare c61 with its twin on the same Sparks 1–4.
- Prefill yield off: 16-user total 1% higher, one-user decode the same, but the short chat waited 1.51 s, worse than r6x. Declined.
- Prefill yield on (
--prefill-yield short): one-user decode identical (123.18 both), 16-user total 1% lower, prompt reading slightly faster, and the short chat dropped to 0.70 s. Promoted. - c61 overlay: Owen's next branch changes how draft tokens are priced and screens the drafter's word list in 8 bits. Every check passed. Against the same configuration without it on the same four Sparks, one-user decode read 123.71 tok/s against 122.72 (+0.8%, inside run-to-run spread), the 16-user total was 2.4% lower, and the short chat waited 0.85 s against 0.73 s (the three runs overlap). Not adopted; worth a retest when he publishes results.
The two fixes we mount
Structured-output crash (spark-ds41f#1194). From 1107 on, the draft model can propose up to 16 tokens to check per request, but the grammar mask for JSON output was sized for 5. A burst of a dozen or so JSON-format requests overruns it and the engine exits, taking every request in flight with it. Zee-B33 diagnosed it and posted a three-line fix; we mount that file unchanged. mjcryptocloud and hxfhd reproduced the crash on their rigs. We did not try to reproduce it; we added a 64-request JSON burst probe to the gates, and every candidate-60 boot passed it with the fix in place.
Tool arguments that quote the model's own tags. DeepSeek-V4.1 writes tool calls in a markup it calls DSML. When an argument itself contains a closing DSML tag (a file about DSML, say), parsers cut it short. Our earlier fix, after an idea from Hector Isaza's TensorFold patch, did not carry over to candidate 60's rewritten parser, so a worker running on our GB300 re-ported it. Stock candidate 60 fails 6 of our 25 synthetic cases; the port passes all 25 inside the image. Stock candidate 60 does pass our live probe, so this fix covers the edge cases, not the common path.
How we decided
We wrote the promotion rule before the first boot: against a same-night control on the same nodes, the candidate had to
- give identical greedy replies across two boots;
- keep qeval, the 1M needle and tool calls at parity, and pass the JSON burst probe;
- keep both mixed-traffic delays within 20% of the control;
- lose no more than 2% on one-user decode, and gain at least 5% somewhere real.
It cleared every line with room to spare. The old r6x containers are stopped, not removed, and the recipe carries a one-command rollback. The lane went back on our guest gateway after the abliterated weights it serves were swapped back in.
What we did not measure
- All speeds are on the base 1017 weights. The lane serves the abliterated variant of the same model, and we have not re-run the battery after the swap.
- Candidate 60 reports room for 12.03M tokens of prompt cache against r6x's 8.58M from the same 10.5 GiB per node. We did not test whether that many tokens can be in use at once, so we do not claim the extra capacity.
- No teacher-forced comparison of token probabilities against r6x, and no measurement of how many draft tokens are accepted.
- Candidate 60 is a candidate, not a frozen release. When Owen tags it we will check the source fingerprint still matches.
- One night, one battery, one switch. Owen's ring and switched testbeds are different hardware and his benches differ from ours; his figures are his.
Sources
- christopherowen/spark-ds41f at
fcc97650: candidate-60 image config and patch series (experiments/2026-11-16-c60-prefill), switched profile, c61 overlay (experiments/2026-11-16-c61-decode). sparknet. - spark-ds41f#1194: the structured-output crash, Zee-B33's diagnosis and fix.
- Our recipe, with all receipts: results/2026-11-16-r7-candidate60 (J-M-Recipes#131).
- Earlier posts: how candidate 60 works and what a GB300 can borrow, four engines on four Sparks.