Dual 12GB vs single 24GB: we ran the same 27B on four machines and the GPU count barely mattered

The most-asked local-LLM hardware question in 2026 is some version of: "Should I buy two 12GB GPUs or one 24GB GPU?" The answers online are bad. They change three variables at once — card, model, and quant — and report a single bar chart that mashes all of it together.

We tried to answer it cleanly. We took one model (Qwen3.6-27B at Q4_K_M), verified byte-identical weights across our fleet (Ollama digest a50eda8ed977), and ran the same prompt under the same conditions on all four lab machines. The full data is in bm-013.

The short answer is boring, and that's what makes it trustworthy: at 27B Q4_K_M, GPU count barely matters. Two cards only start earning their keep once the model stops fitting in one.

The setup

Four machines, one model, one harness:

Machine	GPU(s)	VRAM	Backend
Victor (MSI laptop)	2× NVIDIA RTX 5070 12GB	24 GB combined	CUDA 13.0
EvoX2	AMD RX 7900 XT	20 GB	ROCm 6.17
Ray	2× AMD Radeon AI PRO R9700	64 GB combined	ROCm 7.0
Jitori ⚠️	Intel Arc Pro B60	24 GB	Vulkan (fallback)

Every run used the same ollama_bench.py harness (pure Python stdlib — same code on every host), the same 37-token prompt, 512 generated tokens, temperature=0, seed=42, and one discarded warmup rep. The model is byte-identical on Victor, EvoX2, and Ray — same digest, same weights. Jitori pulled the same tag.

The result

Generation speed (tokens/second, 512 generated tokens):

Rank	Machine	GPU topology	gen tok/s	prompt tok/s	wall (s)	peak VRAM
1	Ray	dual 32GB AMD	26.74	217.27	19.71	21.2 GB / 64 GB
2	EvoX2	single 20GB AMD	25.21	162.05	20.84	16.8 GB / 20 GB
3	Victor	dual 12GB NVIDIA	24.28	344.85	21.57	9.0 GB / 24 GB
4	Jitori ⚠️	single 24GB Intel Arc	8.41	111.94	61.72	19.3 GB / 24 GB

Read the top three rows carefully. Ray, EvoX2, and Victor are within 10% of each other for generation — 26.74 vs 25.21 vs 24.28 tok/s. That spread is inside run-to-run noise. For a 27B Q4_K_M model, ~25 tok/s is just what current-generation discrete GPUs in this class deliver.

Finding 1: dual-12GB is NOT faster than single-20GB at this size

This is the RQ-001 answer, and it's the boring one.

Victor (two 12GB NVIDIA cards, tensor-split across both) generates at 24.28 tok/s. EvoX2 (one 20GB AMD card) generates at 25.21 tok/s. The single card is slightly faster. The model is 17 GB at Q4_K_M; it fits in one 20GB card with room to spare, and it even fits in a single 12GB card (Victor's peak VRAM was 9 GB — Ollama kept most of it on GPU 0). Splitting across two devices adds overhead (inter-GPU tensor copy on every layer) and buys you nothing when the model already fits.

The dual-GPU tax is real and small. The dual-GPU benefit only shows up when the model can't fit in one card — i.e., once you exceed ~18 GB at this quant. Below that threshold, you're paying for a second card and getting slower inference.

The practical version: if your target is 27B-class models at Q4_K_M, one 20–24GB card beats two 12GB cards. Save the dual-GPU budget for 35B+ or for running two models concurrently.

Finding 2: NVIDIA wins prompt-eval by ~2×

Generation speed clusters. Prompt-eval does not.

Victor (Blackwell CUDA) processes the prompt at 344.85 tok/s — roughly double EvoX2's 162 and well ahead of Ray's 217. NVIDIA's batched-prefill throughput is materially better than ROCm's right now, even when steady-state generation is comparable.

This matters for workloads that are latency-sensitive on first token: chat UIs, agents that fire many short prompts, RAG retrieval. If your workload is "one long prompt, long generation" (translation, summarization, long-form writing), the prompt-eval gap barely shows up in the user experience and the generation cluster is what you feel. If your workload is "many short prompts" (agent loops, tool use, conversational back-and-forth), NVIDIA's prefill advantage is worth the premium.

Finding 3: Jitori's 8.41 is not the Arc B60

We're publishing the Jitori number because not publishing it would be dishonest — but we're labeling it loudly: 8.41 tok/s is Vulkan-fallback performance, not the Arc B60's real capability.

Stock Ollama on Jitori has no SYCL runner, so it falls back to a generic Vulkan shim that the community has documented as degraded on Battlemage for ≥20B models. We tried to get the real SYCL number: a genuine llama-server SYCL build, the documented Battlemage env vars (GGML_SYCL_DEVICE=0, GGML_SYCL_USE_LEVEL_ZERO_API=1, GGML_SYCL_DISABLE_OPT=1, --no-warmup, --cache-type-k/v q8_0, --flash-attn on). SYCL detected the B60 correctly, loaded the model into VRAM, and then produced 5.97 tok/s of garbage (#///////////////////////////// repeated, with the GPU reporting 0% busy during inference).

That matches upstream bug ggml-org/llama.cpp#22413 ("Brutally bad SYCL performance on Battlemage"). The documented Battlemage fixes no longer work — likely a regression from a llama.cpp bump or an oneAPI 2026.0 incompatibility.

The good news: OpenVINO Model Server works cleanly on this exact GPU. We benchmarked it in bm-012 — same Arc B60, 30B-A3B class, and it hit 67.95 tok/s, ~1.76× the Vulkan path. So the B60 is a capable card; the SYCL software stack is what's broken. We'll re-baseline Jitori properly once that lands.

What this does NOT tell you

We'd rather say this explicitly than let you over-read the table:

• It's not a clean vendor comparison. Victor is CUDA, EvoX2/Ray are ROCm. The dual-vs-single topology is confounded with the vendor backend. A clean topology test needs dual-vs-single within one vendor, which our fleet can't do.
• It's one rep per machine. Seed 42, temperature 0 — deterministic output — but tok/s varies a few percent run-to-run. Treat the top three as a cluster, not a strict podium.
• It's a single-prompt probe, not our 100-prompt pack. The decision value is the cross-machine comparison (same harness, same weights, same prompt everywhere), not an absolute throughput spec. Don't rank these numbers against our workload-pack-v1 records — different protocol.
• We didn't measure TTFT or power. Those fields are zero in the record. We measured end-to-end wall time and prompt-eval rate; that's it.

The takeaway

If you're buying hardware for local 27B-class inference in 2026, the decision tree is simpler than the forums make it sound:

1. Will your largest model fit in one card? (At Q4_K_M, 27B fits in 20GB; 35B fits in 24GB.) → Buy the biggest single card you can afford. Dual-GPU won't help.
2. Do you run agent loops or many short prompts? → Lean NVIDIA. The prompt-eval advantage shows up there.
3. Are you targeting 35B+ or multi-model concurrent serving? → Now dual-GPU makes sense. This is where Ray's 64GB combined earns its keep.
4. Are you on Intel Arc Battlemage? → Use OpenVINO Model Server, not SYCL. See bm-012.

The raw evidence — meta.json, metrics.json, console.txt, vram_samples.json for all four machines — is in the bm-013 raw archive. Pull it and recompute. That's the point.

---

This is the first cross-machine baseline from the Local AI Research Lab. The next clean comparison is devstral:24b (identical digest on Jitori and Ray — a real Intel-vs-AMD vendor test). Subscribe to the RSS feed or the newsletter to get it when it lands.