The local-AI GPU buyer's guide

Buying a GPU for local AI is not the same as buying one for gaming. The bottleneck is VRAM capacity and memory bandwidth at sustained load, not peak frame rate. This guide walks the decision by workload, with numbers pulled from our lab benchmarks.

The decision, in one question

What is the largest model you want to run comfortably, and how latency-sensitive is it? Everything else — brand, peak TFLOPs, ray-tracing cores — is secondary.

Three honest starting points

Based on our tested builds and benchmarks:

• < $1,500 budget, 7B–13B models, occasional 30B at Q4: Intel Arc B580 12GB on Vulkan. Best $/tok we've measured. raw data
• ~$2,200 budget, 30B daily, 70B at Q3: AMD RX 7900 XTX 24GB on Vulkan. Single-card simplicity, 24GB fits the sweet spot.
• ~$3,500 budget, latency-critical agent loops: NVIDIA RTX 4090 24GB on CUDA. Still the speed leader at this tier. raw data

The comparison table

Every spec you actually need to decide, in one place. Numbers are drawn from our measured benchmarks, not spec sheets — follow the links for the raw runs.

The "fits" column is a Q4_K_M rule of thumb; real headroom depends on context length. Check raw data for the APU ceiling and raw data for the 24GB tier.

Brand-by-brand, honestly

NVIDIA

Still the speed leader and the only mature CUDA ecosystem. If your workload is latency-sensitive (agent loops, voice, real-time coding), CUDA's lead is real — our benchmarks show ~35% faster generation at the same 24GB tier. The cost is money and power: NVIDIA cards are more expensive per GB of VRAM and draw more watts.

AMD

Best raw VRAM-per-dollar at the mid and high tiers. ROCm works but is finicky on consumer cards; Vulkan via llama.cpp is currently the more consistent path for RDNA3–4 hardware. Multi-GPU AMD configs win for throughput workloads but add complexity (tensor-split quirks, driver attention).

Intel Arc

The price-performance surprise. Battlemage (B-series) on Vulkan lands within ~12% of equivalently-priced AMD/NVIDIA on generation speed for code workloads, often at lower system cost. Critical caveat: SYCL produces garbage output on Battlemage — use Vulkan. If you can accept that constraint, Arc is the budget winner.

VRAM tiers and what they actually run

Capacity is the gating constraint — not speed. Here's what each VRAM tier unlocks at Q4_K_M (the sensible default quant):

• 8GB — 7B models only. You will outgrow this.
• 12GB — 7B–13B comfortably; 30B is tight and leaves no context room.
• 16–20GB — 30B daily-driver territory. The practical enthusiast floor.
• 24GB — 30B with full context headroom, or a 70B at Q3 with offload.
• 48GB+ (multi-GPU or dual 24GB) — 70B at Q4_K_M without compromise.

Run your own numbers with Model Fit before buying — the gap between "technically fits" and "runs well" is the whole game.

What to ignore

• Peak TFLOPs — marketing number; irrelevant to sustained LLM inference.
• Ray tracing cores — unused by LLM runtimes.
• "AI performance" on the box — usually FP16 matrix-multiply marketing, not what llama.cpp measures.
• Brand-tier rankings — the right GPU depends on your workload, full stop.

FAQ

How much VRAM do I need for local AI in 2026?

12GB is the practical floor for anything beyond 7B models at Q4. 16-24GB is the useful range that lets you run 30B models at Q4_K_M and leaves headroom for context. 8GB cards trap you at 7B and will be outgrown within months.

Is Intel Arc actually viable for local LLMs?

Yes, with one hard caveat: on Battlemage (B-series) you must use the Vulkan backend. SYCL produces garbage output on BMG G21. On Vulkan, Arc B-series lands within roughly 12% of equivalently-priced AMD/NVIDIA cards for code workloads, at lower system cost.

NVIDIA or AMD for a first local-AI build?

If your workload is latency-sensitive (agent loops, voice, real-time coding), NVIDIA CUDA is still about 35% faster at the same 24GB tier and the ecosystem is more mature. If you want raw VRAM-per-dollar for throughput, AMD RX 7900 XTX 24GB on Vulkan is the value pick.

Before you buy

1. Use Model Fit to estimate what your target model needs.
2. Check our benchmarks for your platform.
3. Read the matching build recipe for parts and total cost.
4. Confirm PSU, case clearance, and cooling — local AI sustains load for hours, not seconds.