← All Topics

DGX Station GB300

Public notes from one NVIDIA DGX Station GB300 in a home lab. Recipes, dated measurements, and the configs that lost. This is the page to share. External coverage of the deskside SKU is collected on GB300 Links.

If you only read one: Testing the inner loop on DSFVE — the evidence trail and the simpler, digest-pinned Recipe v2 generated from it. Recipe, launcher, and sanitized receipts are on GitHub.

Spark (GB10) posts are a different machine. Those live under Local LLMs.

Hardware: the empty CX8 M.2s are filled. Exxact M.2 upgrade — 2× WD_BLACK SN850X 8 TB, RAID0 14.55 TiB at /models. Identify disks by serial; nvmeN reshuffles.

What this Station has actually served

WorkloadWhat is provenRead
DeepSeek-V4.1-Flash adaptive residency, explained October 25, 2026: plain-language overview of the series. Why a 475 GiB model needs a hot/cold expert map on a 250 GiB HBM box; the counter that lets the map follow traffic (+111% on Japanese/German prose, +46% mixed at one user, +82% at four); and the seed map that beat a faster swapper (+44.5% held-out mixed with adaptation frozen). Includes the limits still owed (BFCL, GSM8K, 128K/1M, soak) and the next steps. Explained
DeepSeek-V4.1-Flash sidecar, mixed-traffic map (v25m) October 23, 2026: on the RTX-sidecar lane the GB300 waits on the RTX 11.8% of wall time on Japanese/German prose, 0.5% on agent English. A static map refit from live counts (one file) gives +5.8% held-out mixed C1 (prose +15%), C4 unchanged; wait 11.8% → 0.5%. Experimental: parity cannot gate this lane (same-map self-agreement 14/50); fidelity gates not re-run. Mixed map
DeepSeek-V4.1-Flash adaptive expert residency October 23, 2026: a relaxed-atomic counter inside the existing split kernel plus between-step swaps lets v23's 295 hot experts per layer follow live traffic. Japanese/German prose C1 90 → 189 tok/s (+111%), agent English +12%, mixed traffic +46% C1 / +82% C4; cold share 17–28% → under 1%. Counter 2–6 µs per step; round-trip parity within lane noise. Not yet promoted. Adaptive residency
DeepSeek-V4.1-Flash adaptive residency, seeded map (v26 candidate) October 24, 2026: booting the adaptive lane from the counter's own refit map (top 295 experts per layer, written October 23) is worth +44.5% held-out mixed C1 with adaptation frozen (127.1 → 183.7 tok/s) and 2.1× on the first prose request after boot. Aggressive swap queue rates (2× and 4×) bought nothing: a seeded map peaks at 3–6% cold share instead of 25%, so the queue never saturates. First measured swap cost (~1.9× step p95, 0.26 ms row copy). On a 48-hour soak; not yet promoted. Seed map
MiMo-V2.6-Pro self-blog September 22, 2026, 9:10 PM CDT: asked mimo26-pro to rewrite its own serving post. It wrote each artifact in one near-16K completion against the lane's 16,384 output cap (15,981 tokens in 539 s); Hermes continued 4x then ended both turns as error. Draft validated locally, no upload. Not a remeasure of decode. Self-blog truncated
DeepSeek-V4-Flash-Vision-Exp (Recipe v2) Native FP4, digest-pinned SGLang, DSpark, explicit static verification, no SPS table, checkpoint-default NextN, 1M context, TP=1. Iteration 13 measured 2,985.4 tok/s at C64; C8–C64 was equivalent to baseline inside the frozen ±3% band. Recipe v2 evidence · original post
GLM-5.3-Flash NVFP4 DFlash2 daily-driver recipe on SGLang v0.5.20. ~200 tok/s C1 answer-only (265 if thinking counted); do not use the old 234. BFCL held-out Pass@1 79.3%, needle 27/27 to 881K, GPQA-Diamond 75.3% at a 64K cap. Verify-width patch parked (Sept 23). Warmup is mandatory; tool-call parsers are not optional. living post + recipe card · Flash round 3
Full GLM-5.3 744B NVFP4 K=2 closed on the 512K slot-cache lane (9/20 greedy, 47.13 vs 48.03 tok/s). Keep K=1. Decode remains GPU-bound and dominated by masked_row_copy C2C miss bytes. 744B testing · HBM expert allocation
GLM-5.3 research pass Independent replay reproduces the offline simulator exactly (hit 0.7331112905490802, 4.2702 misses/step/layer). 17.6 of the measured 40.1 ms decode step is expert-row copy over C2C — 44% of the step at 80% of the measured 383 GB/s ceiling. Cache policy is not the lever; KV placement is, and it has never been measured. 1.78× honest ceiling. GLM 5.3 research
GLM-5.3-Flash, round 3 Ruler fix: round-2 C1 rows measured max-effort thinking; answer-only is 202.7 tok/s (265 thinking counted). Nightly 8874c51a new daily; block 7 kept over 5/6; fused verify, ReplaySSM spec, TRT-LLM DSA, native MTP, trtllm MoE closed with receipts. Clean C8 732 agg. Flash round 3
GLM-5.3-Flash, round 2 NVIDIA NVFP4 + DFlash2 block 7, 1M ctx: 252 ~202 tok/s C1 answer-only (corrected, see round 3); KL-gated vs FP8 original (0.136, ~16% token flips either quant). Slot-cap correction: DFlash2 wins per seat through C24. AR lane: 48 users at 50 tok/s each, 128 at 34, TTFT p95 1.4 s. Flash round 2
MiMo-V2.6-Pro (1.02T), recipe v23 Hotsplit Verified September 24, 2026; weights moved to Xiaomi's MOPD checkpoint and re-verified September 28 (flood fixture: RL 25/36 history-primed tasks flooded, MOPD 0/36). 320 GiB of experts in Grace; hotsplit keeps the measured-hot experts in HBM: decode 31.2 → 37.4 tok/s at 11.5K, C8 63.7 aggregate. Against stock placement: top-1 flips 1.23% (reboot floor 1.17%), BFCL held-out 78.2% vs 78.5%. 1M variant: needles 3/3 at 1.04M, 25.7 tok/s there. Day one's garbage was --load-format runai_streamer zeroing fused-qkv fp8 weights. A 1-terabyte model on one box: MiMo-V2.6-Pro, explained
  • We tried Christopher Owen's prefill yield on our GB300: real, small, and not promoted
    The one Spark idea that ports to our GB300 sidecar lane: Christopher Owen's prefill yield. It cut a short chat's first token behind a 413K-token prompt by about 40% on two boots and cost nothing we could measure, but the row we had promised to judge it on did not reproduce, so we rolled it back.
  • Christopher Owen's spark-ds41f at candidate 60: how it works, how it compares to our recipes, and what a GB300 can borrow
    A read of Christopher Owen's DeepSeek-V4.1-Flash stack for DGX Sparks as it reaches candidate 60: how the engine works, his release ladder next to our measured Spark lanes, and which ideas move to a GB300 with an RTX PRO 6000 sidecar.
  • One byte in 250 calls: a 6% faster DeepSeek-V4.1-Flash recipe, and the rounding bug behind it
    Pg62 is our DeepSeek-V4.1-Flash serving recipe for one DGX Station GB300: Garner McCloud's W1 decode-attention kernel now covers 38 more layers, both W1 extensions rebuilt to round exactly like the engine's stock op. +6.1% single-user decode, and zero differing bytes where the kernel we had been serving changed one in 250 calls. Recipe, patches and receipts are public; the lane is still in its soak.
  • DeepSeek-V4.1-Flash on One GB300 Without the Second GPU: What the Sidecar Is Worth
    We took Garner McCloud's two-GPU DeepSeek-V4.1-Flash engine for the DGX Station and ran it on the GB300 alone, with the cold experts copied in from Grace memory. Same prompts, same night: 261 tokens per second at one stream, against 225 for our October single-GPU recipe and 412 with the second GPU. It missed the 1.25x bar we set for it. Recipe and receipts are public.
  • Can One GB300 Station Run DeepSeek-V4.1-Flash in 5 ms a Step?
    The decode-runtime project: a from-scratch runtime for DeepSeek-V4.1-Flash on one DGX Station GB300. It reached 18.6 ms a step (6.4 ms a token) with its arithmetic kept bit-identical, against 7.66 ms on the two-GPU production lane. Corrected November 9: it never beat the single-GPU vLLM recipe we already had (different tests, never run side by side), so the runtime is now a research instrument, not a way to serve. The serving result is in the one-GPU gracetier post.
  • MiMo-V2.6-Pro testing
  • MOPD weights, re-verified (September 28)
  • Muse Spark Open 816B NVFP4 Day-after-release on one Station: 311 GiB experts in Grace (pin-memory off), vLLM day-0 image, DSpark k=2. 29.4 tok/s C1 prose, C4 49.2; BFCL 563 vs 561, GSM8K 193 vs 192 vs no spec. Autotune off: boot 1 h 48 min → 6 min, same speed. k=7 slower than no spec. Muse Spark Open on one GB300
    Step-5-Preview NVFP4 Day-one bring-up on one Station (vendor floor 4× B300): 180 GiB experts on Grace, 256K, 30.1 tok/s C1, 39–42 with MTP k=1 (bit-identical), C8 86.9. Needles 30/30 to 258K, BFCL 559/600. Hot/cold expert split estimated +25–40%, parked. Step-5-Preview on one GB300
    DeepSeek-V4.1-Flash + RTX 6000 sidecar Serving :30006 since October 22. GB300 holds 295 hot experts/layer, the RTX PRO 6000 computes 89 cold ones: 279 tok/s one user (+49%), 3.0× at 16 users, 1M prompt first token 33 s; 12/12 quality gates; fail-closed watchdog. RTX 6000 sidecar
    DeepSeek-V4.1-Flash native Living log of DeepSeek-V4.1-Flash on one DGX Station GB300 + RTX PRO 6000 sidecar at full 1M context. October 31: Pg61 (Garner McCloud's DS4.1 engine under our policy) is the lane and ships as an experimental recipe: 1.31× the old lane at one user, prefill 6–9 % slower; fault injection, rollback drill, second-card numerics, forward fidelity and temperature-1 sampling calibration all pass; 48 h soak ends Monday November 2 at 11:44 AM CST. Flash 4.1 testing
    DeepSeek-V4.1-Flash expert offload 98.7% of routed-expert selections hit 73% of the table; a usage-chosen cold set would see 2.7% vs 24% positional. Closed on hardware: weights are one contiguous tensor per layer (no per-expert kernel hook) and the GPU reads Grace at full speed (ATS, 340 GB/s) or with placement control at quarter speed (90 GB/s). Two-tensor row-map = +20% C1 in-domain, 74-min autotune; not worth it. Offload by expert

    Numbers are dated lab measurements from the linked posts, not leaderboard claims. Different engines, quants, and concurrency are not automatically comparable.

    Posts

    Public recipe repos

    Related