DGX Station GB300
Public notes from one NVIDIA DGX Station GB300 in a home lab. Recipes, dated measurements, and the configs that lost. This is the page to share. External coverage of the deskside SKU is collected on GB300 Links.
If you only read one: Testing the inner loop on DSFVE — the evidence trail and the simpler, digest-pinned Recipe v2 generated from it. Recipe, launcher, and sanitized receipts are on GitHub.
Spark (GB10) posts are a different machine. Those live under Local LLMs.
Hardware: the empty CX8 M.2s are filled. Exxact M.2 upgrade — 2× WD_BLACK SN850X 8 TB, RAID0 14.55 TiB at /models. Identify disks by serial; nvmeN reshuffles.
What this Station has actually served
| Workload | What is proven | Read |
|---|---|---|
| DeepSeek-V4-Flash-Vision-Exp (Recipe v2) | Native FP4, digest-pinned SGLang, DSpark, explicit static verification, no SPS table, checkpoint-default NextN, 1M context, TP=1. Iteration 13 measured 2,985.4 tok/s at C64; C8–C64 was equivalent to baseline inside the frozen ±3% band. | Recipe v2 evidence · original post |
| GLM-5.3-Flash NVFP4 | DFlash2 daily-driver recipe. Published: 234 tok/s single-stream; warmup is mandatory; tool-call parsers are not optional. | Flash testing |
| Full GLM-5.3 744B NVFP4 | K=2 closed on the 512K slot-cache lane (9/20 greedy, 47.13 vs 48.03 tok/s). Keep K=1. Decode remains GPU-bound and dominated by masked_row_copy C2C miss bytes. | 744B testing · HBM expert allocation |
| GLM-5.3 research pass | Independent replay reproduces the offline simulator exactly (hit 0.7331112905490802, 4.2702 misses/step/layer). 17.6 of the measured 40.1 ms decode step is expert-row copy over C2C — 44% of the step at 80% of the measured 383 GB/s ceiling. Cache policy is not the lever; KV placement is, and it has never been measured. 1.78× honest ceiling. | GLM 5.3 research |
| GLM-5.3-Flash, round 3 | Ruler fix: round-2 C1 rows measured max-effort thinking; answer-only is 202.7 tok/s (265 thinking counted). Nightly 8874c51a new daily; block 7 kept over 5/6; fused verify, ReplaySSM spec, TRT-LLM DSA, native MTP, trtllm MoE closed with receipts. Clean C8 732 agg. | Flash round 3 |
| GLM-5.3-Flash, round 2 | NVIDIA NVFP4 + DFlash2 block 7, 1M ctx: |
Flash round 2 |
| DeepSeek-V4.1-Flash native | One Station, native weights, 1M context, DSpark k=5: ~88 tok/s on real agent turns (150 on clean code), 18K tok/s prefill, 972K prompt in 85 s; 8-min boot; Hermes-wired. Not promoted. | Flash 4.1 testing |
| DeepSeek-V4.1-Flash expert offload | 98.7% of routed-expert selections hit 73% of the table; a usage-chosen cold set would see 2.7% vs 24% positional. Closed on hardware: weights are one contiguous tensor per layer (no per-expert kernel hook) and the GPU reads Grace at full speed (ATS, 340 GB/s) or with placement control at quarter speed (90 GB/s). Two-tensor row-map = +20% C1 in-domain, 74-min autotune; not worth it. | Offload by expert |
Numbers are dated lab measurements from the linked posts, not leaderboard claims. Different engines, quants, and concurrency are not automatically comparable.
Posts
- GB300 GLM-5.3-Flash Round 3: I Was Benchmarking the Wrong Thing, Then Seven Axes Closed
GLM-5.3-Flash round 3 on one GB300: the enable_thinking flag was a no-op, so round-2 single-stream numbers measured max-effort thinking with the text counted — re-measured answer-only at 202.7 tok/s (265 with thinking counted). Then eight one-axis windows: nightly 8874c51a is the new daily (+2.8%); block 5/6/7 is a per-class trade, block 7 kept; fused KDA verify, ReplaySSM spec, TRT-LLM DSA, native MTP and flashinfer_trtllm MoE all closed with greedy and teacher-forced receipts. Native MTP now boots on NVFP4 with natural acceptance 3.16 and still loses to DFlash2. Recipe PR #25. - Speculative Decoding by Traffic: v14 Wins the Matched Replay Updated September 15, 2026
Three alternating v14/v13 boot pairs on DeepSeek-V4.1-Flash: v14 gains 18.1% on initial recorded-history replay and 19.5% on identical-request repeats. Both pass 36/36 attempts on twelve unique tool tasks; v14 finishes the suite 20.3% sooner, while v13 retains lower median replay TTFT. Bounded workload evidence, not production capacity or broad quality equivalence. - Teaching the Speculator What a Draft Costs September 14, 2026
Companion to the DeepSeek Flash 4.1 post. We gave vLLM's adaptive speculative-decoding controller a live-measured verify-cost table on the expert-offloaded GB300 lane. The boot table was 12–40% optimistic, the controller used the corrected one, and almost nothing changed: C1 +3%, C2/C4 −6/−8%, tool-JSON +15%, nothing adopted. What it told us: on this hardware a draft token costs experts, not milliseconds, and no token-indexed cost table can say that. Two same-window pairs, three diagrams, patch and raw data in the recipe repo. - GB300 Links September 14, 2026
Annotated roundup of public deskside GB300 / DGX Station coverage: StorageReview's WS300 review, Ahmad Osman on MTS with a Station in the studio, NVIDIA's own pages, and the OEM list. Not our measurements. - One GB300 for a 35-Seat Investment Fund: Seven Serving Configs Against a Real Workload September 13, 2026
A prospect asked whether one DGX Station GB300 can serve 35 seats. Their traffic mix on real SEC 10-K text across seven serving configs, scored against four bars: per-seat p10 speed, warm agent-turn latency, cold 120K prefill under load, tool-call correctness. Qwen3.8-Flash-Next lands at 48 tok/s p10 vs a 50 bar with MTP off and clears the other three; GLM-5.3-Flash DFlash2 clears speed (67) by starving KV. No config clears all four. - GB300 GLM-5.3-Flash Round 2: Which NVFP4, a Wrong Verdict Fixed, and 128 Users on One GPU
GLM-5.3-Flash round 2 on one GB300: NVIDIA's NVFP4 vs the community quant scored against the FP8 original by teacher-forced divergence (0.136 vs 0.147; both flip ~16% of tokens); the September 1 'DFlash2 loses above 16 users' verdict was a 7-request KDA slot cap, not speculative decoding — corrected; DFlash2 block 7 gives ~202 tok/s single-stream answer-only at 1M context (the 252 originally published here counted max-effort thinking text — corrected Sept 16, see round 3); plain autoregressive serves 48 users at 50 tok/s each and 128 at 34 tok/s each with sub-1.5 s first tokens. Recipe PR #7. - Offload by Expert on GB300: The Skew Is Real, the Kernel Is Not
The expert-routing histogram nobody had published for V4.1-Flash: 98.7% of decode selections hit 73% of experts, so the obvious win is usage-aware offload. It fails on two walls — a layer's weights are one contiguous tensor with no per-expert kernel hook, and the GPU reads Grace at full speed (ATS 340 GB/s) or with placement control at quarter speed (90 GB/s). The two-tensor row-map is the only design that gets both, and it costs +20% C1 in-domain and a 74-minute autotune. - GB300 DeepSeek Flash 4.1 Testing
DeepSeek-V4.1-Flash on one GB300 Station at full 1M context with DSpark on, now 89 tok/s single-stream prose (was 82) after night two measured the expert-fetch tax at ~36% of the decode step and moved 12.7 GiB of experts from Grace back into HBM: 140–160 on code and tool JSON, 311 aggregate at C16, 18K tok/s prefill, 10/10 tool calls under the real Hermes harness. Same-window k-sweep, a void util experiment, and the 74-minute autotune tax, all on the tight instrument; formal J&M recipe. The SGLang whole-layer offloader that booted first did 3.3 tok/s. - keys suggested DFlash2 for our GLM-5.3 recipe
Measured result: DFlash2 booted on the one-GB300 UVA path but accepted only 1.5718 tokens per verification step against the frozen 3.0 gate. It was stopped, not promoted. - Recipe v2 from the DSFVE inner loop September 4, 2026
One completed outer-loop pass: 20 inner-loop iteration numbers distilled into a digest-pinned static/no-SPS Recipe v2, with measured equivalence rather than an invented speed win. - DeepSeek-V4-Flash-Vision-Exp on One GB300: Native FP4, SGLang, DSpark, 1M Context September 3, 2026
Current one-Station recipe. Native checkpoint, no requantization. Loop table, cold-prefill ladder, correctness gates, and every config that lost. - Designing an Inference Recipe Generator/Optimizer September 3, 2026
Pre-build design note: two loops, experiment cards, benchmark tiers, and how a Station becomes a nightly recipe lab. Asking for input. - GLM 5.3 Testing: HBM Expert Allocation September 5, 2026
Slot-cache build for full GLM-5.3: CUDA-graph correction, graph-replayed layer measurements, memory-budget failures, exact pinning, and demand-fill milestones. - GB300 GLM 5.3 Testing September 8, 2026
Living page corrected for E1 v2: decode-only attribution is 40.1 ms wall / 43.1 ms aggregate GPU per step; decode is graph-captured and masked_row_copy is ~50% of decode. - GB300 GLM 5.3 Flash Testing September 1, 2026
Flash NVFP4 from first token to 234 tok/s. Warmup discipline, DFlash2, 105k-token prefill in four seconds, and the tool-call bug no throughput bench caught. - DGX Station GB300: Day One September 1, 2026
Arrival, first power, kernel/driver-module trap, and the ten-launch SGLang saga to first token.
Public recipe repos
- jmeadlock/dsf-vision-exp-gb300 — Vision-Exp launcher, smoke, needle, repetition audit, cold-prefill probe.
- J-M-Recipes/gb300-glm-flash-recipe — Flash launch/warmup/bench scripts, throughput CSV, failure ledger, CDMM notes, 744B campaign writeup.
Related
- GB300 Links — public web roundup of the deskside SKU (StorageReview, MTS, NVIDIA, OEMs).
- Local LLM Stack — where this Station sits relative to the Spark pair and the rest of the lab.
- Local LLMs — Spark (GB10) recipes and the broader fleet.
- Benchmarks — methodology and non-comparable result boundaries.
- M3 Ultra 512 Benchmarks — the Apple Silicon 512 GB Studio, a different machine.