Kimi-K3 in NVIDIA's NVFP4 release is 1,499.2 GiB, and 1,426.4 GiB of that is 82,432 routed experts at 18,579,480 bytes each. It does not fit any box here. Three tiers — Station, Grace, eight Sparks — closed with 4.88 GiB of slack, which is no slack at all once one reservation is wrong. Four tiers close with 58.6 GiB: the GB300 keeps attention and the hottest experts, Grace holds a warm band, the Station's RTX PRO 6000 computes a middle band, and all eight Sparks hold the rest at 100 GiB each and run it themselves.
Update, October 30, 2026: the execution plan is published — Kimi K3 NVFP4 on Station, and 8 Sparks. It reorders the next steps below. The Spark tier needs none of the Station's GPUs, so it starts now while the DeepSeek lane keeps serving, and the Station work moves into one 2.5-hour measurement window that buys the number which actually decides the outcome: the Station's own attention time per layer. At the only value ever measured (3.03 ms) this design is a 3.2 tok/s machine; at 0.40 ms it is 15.1.
Next steps, in dependency order
| # | Action | Needs |
|---|---|---|
| 1 | End AP2 on the Station | James's call. Returns 254 GiB of GB300 HBM, 66 GiB of the RTX 6000 and 366 GiB of pinned Grace. |
| 2 | E0 fabric build: switch B, second power feed, 4+4 split | Switch, its own feed, 1 m breakout measured; Lane B down 90–120 min. |
| 3 | E1–E3: topology A/B, Station link, per-call constants | E0. E2 and E3 also need AP2 ended. |
| 4 | E4 sidecar: K3 experts on the RTX 6000 | AP2's code path, K3 experts, an sm_120 SiTU kernel. Pass: cosine ≥ 0.999, ≤ 150 µs per expert call. |
| 5 | E5 first tokens: one layer, then 92 | E4 plus a one-layer harness; all eight Sparks and the whole pod. |
| 6 | E6 parity, then speed | E5. Log-probability envelope first; tok/s only after parity. |
Status
| Rung | Bar | Where it stands |
|---|---|---|
| K3-0 fabric | Switch built, Station↔Spark ≥ 180 Gb/s, constants recorded | Switch arrives tomorrow; the Station is mid-campaign. |
| K3-1 running | Experts on four tiers, coherent text, any speed | Never attempted end to end. Pieces measured: 269 µs for one expert round trip between Sparks, 438 µs for three, 175 µs inside a local Spark bank, ~314 µs a layer for a 92-layer loop. |
| K3-2 correct | Per-layer and log-probability match against a slow reference | Not started. |
| K3-3 fast | 15 tok/s single stream, official quality | Not proven. The Station's own 279 ms/token term is the risk, not the network. |
The ledger
Every byte is from the safetensors headers, not a model card: 82,432 experts × 18,579,480 B is 1,426.36 GiB, and the text non-expert set is 72.03 GiB. Each tier pays its own reservations first; those reservations are assumptions, unchanged since October 3, and the first thing to re-measure when an engine exists.
| Tier | Capacity | Subtracted first | Experts |
|---|---|---|---|
| GB300 HBM | 250.69 GiB (256,703 MiB) | non-expert 72.03 + KV 0.84 + KDA 0.42 + workspace 12.0 | 9,558 · 165.40 GiB |
| Grace | 446.03 GiB obtainable | host 12.0 + staging 4.0 | 24,852 · 430.03 GiB |
| RTX PRO 6000 | 95.59 GiB (97,887 MiB) | context + ring + display 6.0 | 5,177 · 89.59 GiB |
| 8 × Spark | 104.49 GiB ceiling each | OS 11.19 + worker 6.0, then budget 100 | 46,232 · 800.00 GiB |
| Total | 1,485.02 GiB budgeted | 85,819 slots for 82,432 experts |
That is 3,387 spare experts, or 58.6 GiB. The slack absorbs the assumptions instead of resting on them: 128K context instead of 32K costs 2.53 GiB and leaves 56.1 GiB, loading vision costs 0.83 GiB, and a 20 GiB host overhead still closes. Lowering the Sparks to 95 GiB also closes, with 1,075 spare; 90 GiB does not, at −1,237. So 100 GiB is the floor with room. Shutting down desktop software buys nothing measurable: the Sparks run no display manager, and the Station's GNOME, Xorg and mutter total about 0.5 GiB of host RAM.
What crosses the wire
The fabric moves activations, not weights. One expert is 18.58 MB; one activation is about 7 KiB, so a dispatch is roughly 2,600× cheaper than shipping the expert. Sixteen dispatches per layer, 92 layers per token, and every expert stays where it is.
Port arithmetic still forces 4 + 4. Four cages per CRS804: one cage on switch B for the Station's two legs (planned as one per switch; both landed on B), one inter-switch link, two cages left per switch for Sparks. The 400G inter-switch link carries no K3 traffic — the pod talks Station to Spark, one hop on each switch.
The miss tax
Four teams have shipped NVMe expert streaming for K3: SGLang's SSD Expert Pack (17–22.7% hit rate, 22–27 GB of disk traffic per token), llama.cpp's bit-exact streaming PR, Pulsar at 0.7 tok/s on one Spark, SparkLab's NVFP4 repack at 0.16 tok/s. All of it works, and all of it is storage-bound. That is why the disk is not a tier here.
At 13.8 GB/s the Station's RAID0 takes 1,347 µs for one expert. A tenth of the routing on disk is about 200 ms per token — three times the whole budget for 15 tok/s. The RTX 6000 answers in 93 µs because it reads its own HBM and only the answer rows cross PCIe, the same pattern AP2 already runs here for DeepSeek; a published same-session A/B on a DGX Station GB300 measured 1.97× at concurrency 1 and 2.83× at 16 over host memory. NVMe stays as the failure path: if a tier is missing, the model reads from the checkpoint it already has and gets slower instead of stopping.
Grace is the other meter. It is 3.4 TB/s of HBM against 350 GB/s of C2C, so half the expert mass warm costs 425 µs a layer — about 39 ms per token before attention. Placement has to put the hottest mass in HBM and in the 6000's own HBM first.
Not this
- Not one tensor-parallel group. A GB10 is ~273 GB/s with no GPU-direct; the GB300 would wait on it, and sm_103 and sm_121 kernels differ.
- Not KV for GLM-5.3. A TP group replicates MLA KV on every rank. Data-parallel attention is the lever, and it needs no hardware.
- Not a faster Spark TP8. The 4+4 split puts an inter-switch hop inside any cross-switch collective.
What can stop it
- AP2. Still holding 254 GiB of HBM, 66 GiB of the 6000 and 366 GiB of Grace at 17:15 CDT.
- Power. A ~1.4 kW rack on a 1,440 W strip; switch B needs its own feed.
- Cable. The breakout on hand is 1 m; passive 200G is good to about 3 m.
- The Station's serial term. 279 ms/token, mostly 24 attention layers at 7.2 ms. The NVFP4 path is unmeasured, and every rung runs under the 2,200 MHz cap.
Provenance
Device figures read live on October 28, 2026: B300 256,703 MiB and RTX PRO 6000 97,887 MiB from nvidia-smi at 17:15 CDT, Grace from free -b. The byte census (82,432 × 18,579,480 B; 72.03 GiB text non-expert) is from the safetensors headers. The 269 / 438 / 175 / 314 µs and the 279 ms/token term are October 3 measurements. Grace's 53 µs and NVMe's 1,347 µs are arithmetic on measured 350 GB/s and 13.8 GB/s, and say so on the chart. The sidecar's 93 µs and 1.97× / 2.83× are measured on a DGX Station GB300 for DeepSeek-V4.1, not Kimi-K3. Every reservation in the ledger is an assumption, and says so.