#coding

The deep-dive on use-case #1 from '5 things you can do with a local LLM.' A working coding agent — refactors, tests, explains codebases — running on a local model, with your source never leaving your machine. Setup, model choice, where it's good, and where it falls down.

Every coding-LLM roundup ranks models by leaderboard score. Almost none tell you what the score costs in tok/s when you step down a quant tier — which is the only trade-off that matters once the model has to fit on a card you own. Here's the measured table for the coder we run, across three quant tiers, on the same hardware.

On 24GB Arc, Q6_K is the sweet spot: quality recovers to within 2 points of Q8_0 on HumanEval while staying 38% faster and fitting comfortably. Q8_0 barely fits and leaves no room for context. Q4_K_M is the budget pick when VRAM is tight.

SUPERSEDED 2026-07-18 — see bm-009. Provisional Victor numbers in this record were ~3x too slow (real: 172 tok/s gen, 3403 tok/s prompt) and based on an incorrect single-24GB-GPU hardware spec. Retained for history only; do not cite. The original summary read: 'CUDA on the NVIDIA reference produces 52.1 tok/s vs. 38.6 tok/s on Arc Vulkan — a 35% generation-speed lead at the same 24GB VRAM tier.'

Vulkan on Intel Battlemage lands within 12% of dual-Radeon AI PRO R9700 for code generation at Q4_K_M, at roughly half the system cost. Arc is the price-performance leader for sub-$2K builds; multi-GPU AMD wins raw throughput once you accept the complexity.

Re-measurement of the NVIDIA CUDA path that bm-002 reported provisionally. On Victor's dual RTX 5070 mobile config (Blackwell sm_120, CUDA 13.3), Qwen3-Coder-30B-A3B at Q4_K_M generates at 172 tok/s and processes prompts at 3403 tok/s — roughly 3.3x and 5.6x faster than bm-002's provisional figures (52.1 / 612.4). bm-002 is superseded; these are the real numbers.

On a single Radeon AI PRO R9700 (gfx1201, RDNA4), llama.cpp Vulkan with the MTP speculative-decoding head is the only path that activates Qwen3.6's Multi-Token Prediction — delivering 66 tok/s, roughly 2.5x the 27 tok/s class of every other backend. If you don't load the -mtp.gguf draft head, Vulkan, HIP, and Ollama all land within a narrow 26-32 tok/s band and the architecture's main speed advantage is left on the table.

For the Ornith-1.0-35B MoE model (qwen35moe family) on a single Radeon AI PRO R9700, Ollama is the clear winner at 78.3 tok/s - roughly 2.9x faster than llama.cpp Vulkan (27.3 tok/s), which also spilled past the 32GB VRAM limit into system RAM. The HIP backend failed outright: it could not parse the Ornith GGUF (version mismatch). On MoE models without an MTP head, Ollama's expert routing currently beats hand-tuned llama.cpp on this build.