One, Two, or Four DGX Sparks? Model-Size Math for Picking the Right Hardware

Nvidia Rtx

Dgx Spark

Gpu Computing

Distributed Systems

One, Two, or Four DGX Sparks? Model-Size Math for Picking the Right Hardware

Tudor M

7 min read·

Just now

--

Param count × precision tells you whether you need one GB10, a ConnectX-7 pair, or a four-node stack — and why “more Sparks” still isn’t one big GPU.

The picker shows three DGX Spark options: one box, two, or four. The marketing line jumps from 128 GB to 256 GB to 512 GB, and it is very easy to treat that like buying a bigger GPU.

Each Spark still has about 128 GB on the node. The larger tiers are stacks joined by ConnectX-7 — useful only if your weights (plus headroom) need another pool and your stack can shard.

Below is napkin math for the smallest of those three that fits the model you intend to run — with what we’ve measured on one Spark and on a dual stack, and with an honest gap where a four-node load test still needs publishing.

In every case you get one public SSH into node0. On 2× and 4×, the other machines are satellites on the RoCE fabric — no second hostname to juggle.

256 GB / 512 GB means N pools of ~128 GB joined by the fabric, not one flat address space. nvidia-smi on node0 still shows one GB10. A plain single-process serve uses only that node. Workers sit idle until you shard (tensor/pipeline parallel, multi-node vLLM, FSDP, Ray, and friends).

NVIDIA’s ConnectX-7 clustering docs put models in the ~405B-parameter class across two Sparks when you distribute them. Four Sparks, in the DIY world, typically need a switch — NVIDIA Sync’s Cluster Assistant caps direct cabling at three and switch topologies at four. Enverge sells the stack by the hour so you don’t buy QSFP gear first; the memory model does not change.

For day-one dual SSH and torchrun, see the 2× DGX Spark docs, or start from /etc/enverge/cluster.md on a live 2× booking. This piece stops at which size.

Estimate how many gigabytes the checkpoint weights alone need — before KV cache, runtime, or training extras.

Formula (when the model is quoted as “70B”, “120B”, …):

weight_GB ≈ N_B × b

N_B = parameter count in billions (the number in “70B”)

b = bytes per parameter at your precision: – FP16 / BF16 → b = 2 – FP8 → b = 1 – INT4 / 4-bit → b = 0.5

Same idea with an explicit parameter count N:

weight_GB ≈ (N × b) / 1e9

Worked examples (weights only):

That number is the floor. You still need KV cache, runtime overhead, concurrency, and — on GB10 unified memory — a margin under the host memory guard. A planning heuristic we use: treat ~100 GB of weights as the comfortable single-node ceiling before you negotiate context length and sidecars — embedders, rerankers, small routers that share the box, the way compound AI systems stack multiple components. Filling all 128 GB with primary weights is how you get a load that “fits” and a session that doesn’t.

For fine-tune vs inference limits on one box, see What Fits in 128GB?. Full fine-tunes need optimiser and activation memory on top of weights — expect a smaller comfortable param count, or LoRA/QLoRA. Here we stay on: will this checkpoint load with useful headroom on 1, 2, or 4 nodes?

If your weight math (plus honest headroom)…

A 70B model in FP8 (~70 GB) or 4-bit (~35 GB) is a classic single-Spark fit. So are many 120B 4-bit loads (~60 GB weights).

On a single DGX Spark we ran Intel/Qwen3.5-122B-A10B-int4-AutoRound through a tuning playbook (MTP speculative decoding, INT8 LM head, optional hybrid INT4+FP8) — 28.3 → 51 tok/s on short warm prompts, with room for 256K context without KV-cache compression. A “122B” label does not force a second box.

Remember the napkin row: 70B FP16 ≈ 140 GB. That does not fit one 128 GB node with headroom. Options: quantize/FP8 on 1×, or keep bf16/FP16 and shard across 2×.

We measured the dual path on 2026–07–02: Qwen2.5–72B bf16, TP=2, eager, on the NGC stacked-Sparks path (NVIDIA’s playbook; image nvcr.io/nvidia/vllm:26.05-py3). 64 concurrent requests, 8,769 output tokens in 102.6 s → 85.4 tok/s aggregate. That is a floor, not a ceiling — eager mode, small KV. Proof that the second pool is reachable when the weights need it.

120B FP16 (~240 GB) already wants more than a dual aggregate once KV is honest — or you drop to FP8 on 2× (~120 GB weights + cache). Models NVIDIA’s two-Spark story puts near the 405B class, kept at a fat precision, push toward four 128 GB pools (or heavier quantization on fewer nodes).

The 4× tier is bookable at $4.50/hr and advertised as 512 GB aggregate. We have not published a four-node load test the way we documented 1× and 2× above. If your decision hangs on a specific 4-rank recipe, run a tiny NCCL/torchrun check on the instance before the multi-hour download.

Without a distributed launch, you pay $1.65 or $4.50 for one busy node and cold satellites — the same idle trap as serving that 140 GB FP16 70B on node0 alone.

Docker on these instances: --runtime nvidia -e NVIDIA_VISIBLE_DEVICES=all — --gpus all fails (no-cgroups nvidia runtime). vLLM-style defaults that grab most of “GPU memory” can trip the host memory guard on GB10 unified memory; plan a conservative utilization cap (our dual bf16 run needed enforce_eager around util ~0.68 to stay under the host memory floor).

If your script cannot shard yet, stay on 1×, finish single-node bring-up, then come back.

A — 70B FP8 coding agent, modest concurrency. Weights ~70 GB. 1×, with room for KV and a sidecar embedder. 2× buys unused fabric.

B — Same 70B, bf16 required for an eval that forbids FP8. Weights ~140 GB. 2× with TP=2 (or wait and quantize). This is the dual measurement case above.

C — 120B FP16 research serve, or a ~300B+ checkpoint you will not INT4. Math clears a dual aggregate or sits on its edge once KV is honest. 4× is the size answer — or revisit precision on 2×. Budget a fabric check before the download until a four-node load test ships.

Pricing · What Fits in 128GB? · 2× DGX Spark docs

Disclosure: We are Enverge and offer DGX Spark stacks by the hour. The 1× and 2× figures above come from our notes and stack runs; the 4× size guidance is product + math with a published four-node load test still to come.

When should I quantize instead of stepping up to 2× or 4×? When the same model fits one (or two) nodes at FP8 or 4-bit with the headroom you need — e.g. 70B FP16 (~140 GB) wants 2× or FP8/INT4 on 1×. Prefer the smaller tier if your eval/serve path accepts the precision.

Is 256 GB or 512 GB one big VRAM pool? No. Aggregate unified memory across 2 or 4 GB10 nodes over ConnectX-7. Shard to use more than one pool.

Why does nvidia-smi show only one GPU on a multi-Spark stack? You are on node0. The other GPUs are on satellite nodes over the fabric.

Have you tested all three tiers? 1×: Qwen3.5–122B INT4 AutoRound path (28.3 → 51 tok/s). 2×: Qwen2.5–72B bf16 TP=2, 85.4 tok/s aggregate floor (2026–07–02). 4×: bookable tier; a published four-node load test is still outstanding.

Does this cover training? Training usually needs more memory than inference for the same param count. Use What Fits in 128GB? for fine-tune limits; this piece picks hardware size for loading and serving-shaped headroom.

Should I always take the bigger tier “just in case”? No. Idle satellites are expensive. Clear the math, then pick the tier.