Run a 30B model on a $300 GPU

The fastest myth to kill in local AI is "you need a $1,500 GPU to do anything real." You don't. A 30B-parameter model at Q4_K_M fits in roughly 16.5GB and runs on a $300 card if you pick the right one. This post is the recipe.

What "30B at Q4" actually costs in memory

From the VRAM math in our Model Fit guide:

weights       = 30 × 0.55         = 16.5 GB
kv_cache      = 8.192 × 0.18      = 1.47 GB
runtime       =                   1.0 GB
-----------------------------------------
total                            ≈ 19.0 GB

That's the full 8K-context footprint. Drop context to 4K and you're under 18GB. The point: 30B isn't a 24GB-only tier the way people assume. With quantization and a reasonable context window, 12GB cards become playable — tight, but playable.

The card that makes it work: Intel Arc B580 12GB

Not the card you expected. Two years ago Intel Arc was a punchline. The Battlemage (B580) changed that:

• 12GB VRAM for ~$300 — no NVIDIA or AMD card matches that $/GB at this tier.
• Vulkan via llama.cpp is genuinely fast — within ~12% of equivalently-priced alternatives on generation speed.
• Power efficient — 190W TDP, runs in most existing systems without a PSU upgrade.

One critical constraint: use Vulkan, not SYCL. SYCL produces garbage output on Battlemage. Set GGML_VK_VISIBLE_DEVICES correctly if you have a dual-GPU system so the runtime picks up the Arc, not an integrated AMD GPU. This is documented in our backend guide.

The recipe

1. Card: Intel Arc B580 12GB
2. Model: Qwen3-Coder-30B-A3B, Q4_K_M GGUF
3. Runtime: llama.cpp (latest), Vulkan backend
4. Flags: -ngl 99 -c 8192 -t 8 (full offload, 8K context, 8 threads)
5. Driver: Mesa ANV (the mature Vulkan path on Linux)

If you're on Windows, the Mesa note doesn't apply — use Intel's Vulkan driver. The numbers shift slightly but the recipe holds.

What you actually get

We ran exactly this configuration in bm-002. The headline numbers on the Arc B580 build we call "Jitori":

• ~18 tok/s generation — fast enough to feel interactive, not fast enough for real-time agent loops.
• ~18.1 GB VRAM used — right where the estimate said it'd be, with ~6GB headroom on a 24GB card (or about 2GB shy on a 12GB card, which means dropping context to 4K).
• Quality within ~3 points of Q8_0 on HumanEval pass@1, per our quantization sweep.

The honest read: this is a daily-driver coding assistant configuration, not a high-throughput inference server. It's excellent for one developer doing real work. It will fall over if you try to serve a team.

When to spend more

The B580 is the right answer up to a point. Cross any of these lines and step up:

• You need 70B daily. Step up to a 24GB card (RX 7900 XTX or RTX 4090).
• You need real-time agent loops. CUDA's ~35% speed lead at the same tier matters here — see bm-001.
• You're serving multiple users. You're now building an inference server, not a workstation. Different problem.

Full part lists for each tier are in Builds.

The takeaway

The "$300 floor" is real. If you've been waiting because you thought local AI required a four-figure GPU, the wait is over — the constraint that used to keep people out (VRAM price) is the thing Battlemage specifically attacked. Buy the B580, follow the recipe above, and you'll have a working local coding assistant this weekend.

Want to check whether your target model fits before buying anything? Plug it into Model Fit.