Best GPUs for local AI in 2026: measured cross-vendor

Every few months a new "best GPU for local AI" roundup lands. They rank the right cards — RTX 4090 at the top, RTX 3090 as the used-market value pick, an RTX 4060 Ti 16GB for budget, an AMD honorable mention, Intel Arc dismissed in a sentence. The lists are correct. They are also nearly useless on their own, because not one of them ran the same model file on Arc, AMD, and NVIDIA with sha256-verified artifacts and reported the actual tok/s gap. We did.

This is that comparison — five cards across four vendors, measured on identical weights, with the receipts in the benchmark library. The headline finding is uncomfortable for anyone who wanted a clean winner: the cross-vendor gap is real (~3.9×), and so is the price gap (~2× in the other direction), and which one matters depends entirely on the workload.

The five cards we actually measured

GPU	VRAM	Vendor	Backend	Fleet machine	Approx street price
Intel Arc B60 Pro	24GB	Intel	Vulkan / OpenVINO	jitori	$300–$400 (B580) to $700 (B60 Pro)
AMD Radeon RX 7900 XT	20GB	AMD	ROCm 7.2.2	evox2 (discrete)	$650–$750
AMD Radeon AI PRO R9700 (RDNA4)	32GB	AMD	ROCm 7.0 / Vulkan	ray (×2)	$1,000+
NVIDIA RTX 5070 (×2, Blackwell)	12GB each, ~24GB split	NVIDIA	CUDA 13.0	victor	$1,200+ the pair
AMD Ryzen AI MAX+ 395 iGPU	~96–124GB unified	AMD APU	ROCm 7.2.2 (gfx1151)	evox2 (iGPU)	bundled with the APU

Two things to flag up front. First, what's missing: we have not measured the RTX 4090, RTX 3090, Arc B580, or RX 7900 XTX. They appear in our build tiers but not in the benchmark library. Second, the RTX 5070 result is a dual-GPU tensor-split configuration, not a single card — and as you'll see, that distinction matters a lot.

The cross-vendor gap, measured

The cleanest cross-vendor comparison in our library is bm-007: LFM2.5-8B-A1B at Q4_K_M, the exact same sha256-verified GGUF, run on the RX 7900 XT and the Arc B60 with the same llama.cpp flags.

GPU	Prompt tok/s	Generation tok/s	VRAM
RX 7900 XT (gfx1100, ROCm)	7464.85 ±139.70	269.01 ±0.22	6.81 GB
Arc B60 (Battlemage, Vulkan)	1688.53 ±4.55	68.79 ±0.27	4.79 GB

That is a ~3.9× generation gap and a ~4.4× prompt-processing gap, on identical weights, with the same -p 512 -n 256 -ngl 99 -r 3 flags. The RX 7900 XT is the faster card by a wide margin. No qualifier needed.

The uncomfortable follow-up question: is that gap a GPU effect or a model-architecture effect? LFM2.5-8B-A1B is a Mixture-of-Experts model with only 1B active parameters, which is a weird shape. So we ran the same cross-vendor test on Llama-3.1-8B-Instruct — a vanilla dense 8B — and got the same ratio:

GPU	Generation tok/s
RX 7900 XT	105.22 ±0.03
Arc B60	27.01 ±0.00

Same ~3.9× ratio. Same sha256-verified artifact on both cards. The gap is the GPU, not the model. Read about why this matters in the MoE shift — active parameters, not total, are what determine tok/s.

The prompt-processing vs generation split

Generation tok/s is what most people quote, but it is not the only number that matters. For chat workloads, prompt-processing speed (how fast the card digests your input before the first token) dominates perceived latency. For batch workloads, generation dominates throughput.

bm-013 — our first four-machine baseline on identical Qwen3.6-27B Q4_K_M weights — split these cleanly:

Machine	Prompt tok/s	Generation tok/s	Wall-clock (37+512 tokens)
victor (2× RTX 5070, CUDA)	344.85	24.28	21.57s
ray (2× R9700, ROCm 7.0)	217.27	26.74	19.71s
evox2 (RX 7900 XT, ROCm 6.17)	162.05	25.21	20.84s
jitori (Arc B60, Vulkan fallback)	111.94	8.41 ⚠	61.72s

The dual-RTX-5070 rig wins prompt-eval by ~2× and loses generation by a hair. For a chat UI where the user types and waits, the NVIDIA rig feels faster. For unattended batch jobs where you walk away, the single bigger AMD card ties it. The "fastest GPU" depends on whether you're sitting in front of it.

The jitori row needs an explicit caveat: that 8.41 tok/s is stock-Ollama Vulkan fallback, not the Arc B60's real capability. A SYCL rebuild produced 5.97 tok/s with garbage output (matches upstream bug llama.cpp#22413). The Arc B60's real numbers on this workload live in bm-001 — 38.6 tok/s on a 30B coder — measured through llama.cpp Vulkan directly, not Ollama's fallback path.

Where Arc belongs

The Arc B60 is not a fast card. It is, however, a cheap 24GB card — and for the 30B-class models that are the practical sweet spot of local coding work, "cheap and 24GB" beats "fast and 12GB." bm-001: Qwen3-Coder-30B-A3B at Q4_K_M runs at 38.6 tok/s generation on the Arc B60, with 18.1 GB VRAM used and 245W wall power. The same model on a single R9700 hits 43.8 tok/s — marginally faster, on a card that costs roughly twice as much.

For a card you can buy new for $300–$400 (B580) or $700 (B60 Pro), 38 tok/s on a 30B coder is comfortably above reading speed. The full retrospective — drivers, backends, where the ecosystem stands a year in — is in Battlemage one year in. The short version: Arc is the price-performance leader for sub-$1K builds if you accept slower generation, and it gets a real speedup from OpenVINO Model Server (1.76× over Vulkan, see bm-012 and OpenVINO beats Vulkan on Arc B60).

Where dual AMD belongs

The Radeon AI PRO R9700 is a 32GB RDNA4 card, and our Ray machine has two of them — 64GB of VRAM in one box. That pool is what lets you run a 35B-class model at Q4_K_M without spilling into system RAM. bm-005: Ornith-1.0-35B at Q4_K_M fits in 23.2 GB on a single R9700 via Ollama and hits 78.3 tok/s. The same model via llama.cpp Vulkan spilled to 34.2 GB and dropped to 27.3 tok/s — slower because it was paging system RAM, not because Vulkan is inherently worse.

The pattern: once your model exceeds one card's VRAM, multi-GPU stops being optional. Below that line, the second card idles (we measured this directly — Ray's second R9700 sat idle through every workload below 32GB in bm-001). The honest version of "is dual GPU worth it" is in our single vs dual GPU analysis — short answer, only past ~18GB of model weight.

Where unified memory belongs

The most interesting card in our fleet is not a discrete GPU at all. The Ryzen AI MAX+ 395's integrated Radeon 8060S sees the APU's full ~96–124GB of system memory as device memory — no VRAM ceiling in the traditional sense. bm-011: the same LFM2.5-8B-A1B Q4_K_M that hits 269 tok/s on the discrete RX 7900 XT hits 150 tok/s on the iGPU. Slower, yes — but on a machine that can also hold a 70B model in "VRAM" without spilling, because there is no separate VRAM pool to spill from.

Unified memory flips the VRAM math that bounds every discrete card here. If your workload is "fit the biggest possible model," an APU with 96GB+ of unified memory beats any consumer GPU you can buy. If your workload is "fastest possible tok/s on a model that already fits," the discrete card wins. The Model Fit estimator does the VRAM math for both topologies.

The honest caveats

What this article did NOT measure, in plain terms:

• RTX 4090, RTX 3090, RTX 4080, RTX 5090 — not in the fleet. Any claim about them is inferred from architecture, not measured.
• Arc B580 — it is our budget build tier but unmeasured. The B60 data is the closest Battlemage proxy.
• RX 7900 XTX — mid-AMD build tier, also unmeasured.
• Apple Silicon (Metal) — zero data; explicitly scoped out.
• Image and video generation — every number here is LLM inference. Image gen (Flux, SD) and video gen (Wan, Hunyuan) have different bottlenecks and we have not characterized the fleet for them.
• Idle and sustained power curves — only bm-001 and bm-003 report real wall-power. See quantization in watts for the narrow slice we have measured.
• Multi-user concurrency / throughput under load — every result here is single-stream or llama-bench. vLLM/SGLang continuous batching is a different measurement entirely; we have not run it.

Two methodology warnings that apply to every cross-vendor claim above: our workload-pack-v1 (bm-001, bm-002, bm-003) is a 100-prompt HumanEval suite and is not comparable to the llama-bench single-prompt numbers in bm-004 through bm-013. Do not divide one by the other. And bm-002 is superseded — its provisional CUDA numbers were ~3× too slow and based on wrong hardware; cite bm-009 instead.

The decision, compressed

You are...	Use
Buying new under $700, want a 30B coder at usable speed	Arc B60 Pro — 38.6 tok/s on Qwen3-Coder-30B (bm-001). Run a 30B on a $300 GPU.
Buying new $650–$800, want fastest single-card tok/s	RX 7900 XT — 105 tok/s on Llama-8B, 269 tok/s on LFM-8B (bm-007, bm-008).
Need to run 30B+ without spilling, $1K+ budget	Single R9700 (32GB) or dual R9700 (64GB) — Ornith-35B at 78 tok/s (bm-005).
Want lowest chat latency on a model that fits in 24GB	Dual RTX 5070 wins prompt-eval ~2× (bm-013). But a single 24GB card is simpler if the model fits.
Need to fit a 70B-class model	Ryzen AI MAX+ 395 APU — unified memory removes the VRAM ceiling (bm-011). Slower per-token, but it runs.
Want the used-market value king	RTX 3090 (24GB) — we have not measured it, but the architecture and VRAM make it the consensus pick. Cite with attribution.
Want maximum single-card speed regardless of price	RTX 4090 — same caveat. Not measured by us.

The takeaway

The cross-vendor gap is real: AMD's RX 7900 XT is ~3.9× faster than Intel's Arc B60 on identical weights, and NVIDIA's Blackwell wins prompt-eval by another ~2× on top of that. But the price gap is also real, and for the 30B-class models that are the practical local-coding sweet spot, "slower but it fits, and the card costs half as much" is the honest recommendation for most buyers. Pick the card for the workload and the budget, not the brand — and run the Model Fit estimator before you buy.

The five build tiers — $1,200 to $4,200 — with tested part lists are in Builds. The raw benchmark records behind every number above are in the benchmark library. The methodology — what we measured, how, and what we deliberately did not — is on the methodology page.

Last verified: July 2026. GPU pricing and the model carousel move fast — re-validate against current street prices and the latest benchmark records before buying.