A kitchen NSF chrome wire rack, a four-port 400G MikroTik, and one 200G DAC per DGX Spark. That is a working Ethernet/RoCE cluster without a 19-inch cabinet or an NVIDIA Spectrum switch. This page is the current unit — eight Sparks, one per shelf, on a sideways 12×18.
Status. Eight one-wide is the packing. Five extra SI-S1218C decks (three more than the five-node BOM). 54″ kit poles, cut to ~40.4″ — do not buy taller posts. Every CRS804 port is a 2×200G breakout; no Station hop on this switch. All eight Sparks will be undervolted to save power; that is the operating plan, not a measured wattage. CAD is packaging, not a load or thermal sign-off. Do not back the gold intake into a wall.
8
Sparks on this unit
4×400G
CRS804, all spent
8×200G
breakout leaves
20 A
circuit; undervolt planned
10
wire decks
What it is
GhettoRak is an open chrome wire tower. Device fronts face the operator. All I/O exits the rear. Everything sits on the mesh: Sparks, switch, bricks. No extra fans. NVIDIA's own Spark is self-cooled; the rack's job is not to recirculate exhaust into intakes, and not to steal the 2 cm side zone. Eight nodes does not inherit a four-node “no extra fan” conclusion — revalidate once it is loaded.
The frame is a Shelving Inc SC121854-5 12″d × 18″w × 54″h kit assembled sideways so 18 inches is depth. The CRS804 is 387 mm deep including PSU handles; it needs that axis. Five extra SI-S1218C decks make ten shelves. Posts are cut to ~40.4 in after dry-fit — about 13.6 in of scrap from the 54s. Threaded 3″ rubber casters (SI-ARC3WB-4) replace the leveling feet — stem casters are the wrong part. SI posts 64″ and taller ship in two pieces; this stack does not need them.
Two Sparks will not sit side by side on this 12″ unit (150 + 40 + 150 mm > 305 mm). One Spark per shelf is the packing rule here.
This stack
| Clip ring | On the wire | Clear |
|---|---|---|
| 37 / 33 / 29 / 25 / 21 / 17 / 13 / 9 in | One DGX Spark per shelf, 150 × 150 × 50.5 mm, 1.2 kg. Fronts forward, QSFP rear. | 4 in pitch → ~42 mm air between Sparks. |
| 6 in | MikroTik CRS804-4DDQ-hRM, 218 × 387 × 44 mm. ~10 mm per side inside the posts. | 3 in bay, ~21 mm air above the lid. Packaging, not thermal approval. Nothing on the lid. |
| 1 in | Eight ADP-240LB bricks in two rows of four on the 35 mm edge; two 4-outlet strips along depth; eight ThirdReality Gen3. | 5 in bay, ~19 mm over the 99 mm brick. USB-C and C6 stay on the 99 mm faces, not the top. A 12″ width cannot take eight Gen3 across. |
Envelope 305 × 448 × 1129 mm with casters. ~9.6 kg of Sparks on a 12 in tower — a stiff DAC tug can walk it. Brakes on, wall tether under Shelving Inc's 48 in rule. Parametric OpenSCAD: inference-box-ghettorak.scad (mm).
ADP-240LB USB-C is captive 1800 mm. Dress brick tail → rear trunk → up a post → Spark rear PD. Dressed run about 0.6–1.5 m, so the leads reach; lower decks need a coil, not a shorter brick. Do not cut the captive cables. Brick C5 mains stay on the 1″ deck.
Nameplate 8×240 + 123 ≈ 2.04 kW. Realistic draw is not measured at eight. A 15 A / 120 V circuit is 1.8 kW — not enough headroom at nameplate. Use 20 A. All eight Sparks will be clock-capped to save power; that is the intended operating mode. No wattage saving is claimed until it is measured on this stack.
Power (not a desktop undervolt)
GB10 is not Jetson. There is no nvpmodel. nvidia-smi -pl is typically N/A / “not supported in current scope.” NVIDIA’s own split is 240 W brick, 140 W SoC (CPU+GPU), 100 W for CX-7 / SSD / USB-C. nvidia-smi wattage is the GPU rail only, not the wall.
The community method that actually sticks is a max GPU clock lock, not a millivolt offset:
sudo nvidia-smi -pm 1
sudo nvidia-smi -lgc 0,2200 # try 2200 first
# sudo nvidia-smi -rgc # reset to defaultMake it survive reboot with a oneshot systemd unit after nvidia-smi -pm 1 / nvidia-persistenced (same pattern as the NVIDIA forum write-up). Start at 2200 MHz. Published single-stream vLLM numbers on two GB10s (gb10-clock-cap): 2200 vs stock ~2455 MHz was −12 °C peak, −36% GPU-rail power (mean 63.1 W → 40.1 W per node), decode within ~1%, prefill a few percent slower. 2000 MHz was 28.6 W GPU-rail mean. Those are GPU-rail, single-stream, not this eight-node wall.
We have not measured this stack. The table is arithmetic on published numbers plus NVIDIA’s nameplate split. Wall peak on eight nodes with DACs plugged in is unknown until a PDU or kill-a-watt run.
| Budget | What it is | 1 Spark | 8 Sparks + CRS804 |
|---|---|---|---|
| Nameplate | NVIDIA 240 W brick × 8 + switch 123 W | 240 W | 2043 W |
| 15 A / 120 V | Circuit ceiling | — | 1800 W — below nameplate |
| 20 A / 120 V | Circuit ceiling | — | 2400 W |
| GPU rail, stock mean | gb10-clock-cap, two-node vLLM | 63.1 W | 505 W GPU only |
| GPU rail, 2200 MHz mean | same paper | 40.1 W | 321 W GPU only |
| GPU rail, 2000 MHz mean | same paper | 28.6 W | 229 W GPU only |
| Naive wall if rest stays 100 W | 40.1 W GPU + 100 W CX-7/SSD/USB × 8 + 123 W | — | 1244 W — not measured |
Clock-cap only bites the GPU rail. CX-7 with DACs plugged in, SSD, and the CRS804 do not shrink with -lgc. Do not read 8 × 40 W as the strip load. A 15 A breaker is still the wrong plan until a measured wall peak on this eight-node fabric says otherwise. Test stock vs 2200 vs 2000, decode and long prefill, all eight up, DACs in.
Do not confuse a cap with the USB-C PD wedge (GPU stuck ~611 MHz / ~13 W). That is a brick-negotiation bug. Cold-drain the ADP-240LB, keep the supplied 240 W adapter, update PD firmware. Pulling QSFP saves ~18 W via CX-7 hotplug — useless on a fabric that needs the DACs plugged in.
Network
Each Spark has two QSFP cages on a ConnectX-7. Both cages share about 200G of PCIe into GB10. NVIDIA's switch playbook is one cable per Spark. Plugging the second cage does not make 400G.
The CRS804 has four QSFP56-DD 400G ports. Each splits to 2×200G with a QSFP-DD → 2× QSFP56 breakout. Eight Sparks spend every port. MikroTik will not auto-negotiate 2×200G — force 200G-baseCR4 with auto-neg off on lanes 1 and 5 of each split port.
| Sparks at 200G | CRS804 ports used | Left over |
|---|---|---|
| 4 | 2 as 2×200G | 2 native 400G |
| 5 or 6 | 3 as 2×200G | 1 native 400G |
| 8 — this unit | all 4 as 2×200G | none. No Station hop on this switch. |
Cables are NADDOD MikroTik-coded Q2Q56-400G-CUx (400G → 2×200G). Not Q4Q56 (4×100G). Not Generic coding — EEPROM on the QSFP-DD end is the point of paying for the MikroTik SKU. 10G RJ45 on the Sparks and the CRS804 stays management. Do not put SSH on the fabric.
| Qty | NADDOD | Length | Goes to |
|---|---|---|---|
| 1 | #103561 Q2Q56-400G-CU0-5 Mikrotik | 0.5 m | Spark 1 + Spark 2 (9″ + 13″) |
| 2 | #103562 Q2Q56-400G-CU1 Mikrotik | 1 m | Spark 3+4 (17″+21″), Spark 5+6 (25″+29″) |
| 1 | #103563 Q2Q56-400G-CU2 Mikrotik | 2 m | Spark 7 + Spark 8 (33″ + 37″) |
A DGX Station GB300 is a different cage (QSFP112 vs QSFP56-DD) and is not a DAC — two DR4 modules plus an MPO jumper. Not on this CRS804 while it is full of Sparks.
Buy
| Qty | What | SKU |
|---|---|---|
| 1 | 12″d × 18″w × 54″h chrome wire, 5 shelves. Assemble sideways. Cut posts after dry-fit to ~40.4″. Do not buy 64″+ posts for this height. | SC121854-5 |
| 5 | Extra 12″d × 18″w chrome shelf + clips (ten decks total) | SI-S1218C |
| 1 | 3″ rubber threaded swivel casters with brakes, 4-pack. Not stem casters. Not the 4″/5″ pack on a 12″ tower. | SI-ARC3WB-4 |
| 1 | MikroTik CRS804-4DDQ-hRM | CRS804 DDQ |
| 4 | MikroTik-coded QSFP-DD → 2× QSFP56 200G DAC: 1× 0.5 m, 2× 1 m, 1× 2 m | NADDOD Q2Q56 |
| 8 | ThirdReality Gen3, one per Spark brick | MHR 001 |
| 2 | Grounded 4-outlet strips along depth. CRS804 keeps its own IEC inlets. | — |
Spark USB-C is captive on the included brick. No extra power cable. Eaton / UPS stays on the floor. 20 A circuit. Undervolt the Sparks; measure before you trust a 15 A breaker.
Past eight
The switch sets the ceiling. Four 400G cages → eight 200G Spark links. Past eight you buy another CRS804 (or a real 200G leaf) and you stop pretending this is one desk appliance.
Siting is the same at any count: 10 cm in front of the gold mesh, 40 cm behind the QSFP face, open sides. Rear 40 cm is an NVIDIA Quick Start number this desk stack does not provide — open wire plus a DAC canyon is the mitigation. Do not also steal front mesh or the per-device 2 cm side zone.
Printed clips
Optional. The rack works empty. A 1.2 kg Spark plus a stiff 28 AWG breakout will walk on chrome wire, so keepers are the first print once the SI mesh is in the building. Measure wire diameter and pitch on the actual shelves — MakerWorld wire clips are not one size. PETG, H2C, 0.2 mm, no support if the clip is designed for it. Do not block the gold intake.
| Job | Inspiration | Note |
|---|---|---|
| Shelf labels (Spark 1–8, CRS804) | Clip on wire rack label | Snaps on the front wire. Scale if the SI rail is not the photo. |
| Hang DAC / USB-C slack off a post or rear wire | Wire shelving hooks | PETG, Metro/HDX ¼″ and light-duty 3/16″ sizes. Standard Digital File License — link it, do not remix or rehost. |
| Spark / brick stoppers that bite the mesh | Clip-in divider/stoppers | CC-BY-SA. Comments are honest: it fits none of several racks until you measure the gap and scale ~101–102%. Amazon Basics needed 27 mm. SI mesh in CAD is 20 mm — verify on the kit. |
The CAD still has optional L-tabs. A clip that locks to the wire is the better keeper on a mesh deck. Print after calipers, not before.
Do not buy
- Taller SI posts (64″ / 72″ / 84″ / 96″) for this stack. 54″ cut to ~40.4″ is enough. 64″+ ship in two pieces.
- NVIDIA InfiniBand OSFP splitters — wrong cage for CRS804 QSFP-DD.
- 400G QSFP112 DACs for the Sparks — that is a Station/CX-8 part. Spark is 200G QSFP56.
- Four 0.5 m Y-cables for this 37″ stack — the top pair does not reach. The top pair is 2 m.
- Generic-coded breakouts to save $29 on a MikroTik switch.
- A DAC from the Station to the CRS804. Different cages, and this switch is full.
- Extra 200 mm fans, 10-inch wall cubes, or a UPS inside this frame.
When the kit lands: dry-fit posts and rings, switch on 6″ with fans/IEC toward the operator, bricks on 1″, Sparks last, DACs last. Loose straps on twinax, not zip ties.