The folklore says agentic coding is mostly a software problem: pick a harness, plug in any decent local model, and the loop will run. The measured picture is narrower. We have solid numbers for how local models write code in one shot, thin evidence for how they behave on step 7 of a tool loop, and a demand corpus full of people hitting exactly that wall.
Four threads in our corpus ask the same question from different angles. One reader wants the “Best agentic coder model I can fit in 40gb vram?” Another, a self-described complete beginner, asks whether “Qwen3.6-27B + pi.dev” is the right starting point for agentic coding. A third asks, plainly: “What are the agentic softwares you run on top of local llm which can do tasks for you?” And a Hacker News discussion on agent guardrails — 687 points — shows how much anxiety surrounds letting any model, local or hosted, act in a loop at all.
The questions have moved on. Nobody in these threads is asking whether local models can code; our benchmark lab settled that. They are asking which models keep working when the model has to drive — call tools, read output, recover from its own mistakes, and stay inside a VRAM budget while doing it. That is a different question, and it needs a different answer.
Why agent loops break chat models
Chat is one exchange. An agent session is dozens, and three differences compound.
Tool-call discipline. Every step must emit output the harness can parse: a valid tool name, well-formed arguments, an exact file path. A malformed call does not just fail one step — it injects garbage into the context that every later step reads. Chat never tests this, because in chat a malformed answer is merely a bad reply; in a loop it is a corrupted input.
Context accumulation. In chat, the context is mostly what you typed. In an agent loop, the bulk of the context is the model’s own tool output: file dumps, command results, diffs, stack traces. Context grows monotonically, and the KV cache grows with it. Weight quantization — the first lever everyone reaches for — shrinks the weights and leaves the KV cache untouched, so the memory cost of an agent session grows along exactly the axis quantization cannot fix. Our quantization guide walks through that mechanics in detail. Behavior degrades as context fills, too, which is why a local model can feel dumber in a long session than its benchmark scores suggest.
Error recovery. The most agent-specific skill is also the least measured. A command fails; the model has to read the traceback, form a hypothesis, change one thing, and retry. One-shot code generation never tests this. It is the difference between answering a question and debugging your own answer.
Put the three together and you get the pattern practitioners describe: a model that looks fine in chat can fall apart at step 7 of a tool loop — not because it got dumber, but because the loop keeps handing it tasks chat never does.
The measured anchors we do have
The coding leader. Our coding benchmark (bm-003) puts Qwen3-Coder-30B-A3B at the top of the local field: HumanEval pass@1 of 0.86 at Q8_0 and 0.81 at Q4_K_M, running at 38.6 tok/s at Q4_K_M on an Arc B60 Pro while using 18.1 GB. Full results and the rest of the field are in our local coding model guide.
What quantization costs. The step from Q8_0 to Q4_K_M costs five points of pass@1 — 0.86 down to 0.81 — for a large saving in weight memory. Whether that delta compounds, shrinks, or grows inside a multi-step loop is precisely what our data cannot yet say; more on that in the caveats below. The trade-off mechanics, including why the KV cache does not shrink with the weights, are in the quantization guide.
Throughput — and why MTP matters most in loops. Our MTP measurements (bm-004) ran Qwen3.6-27B-A3B-Instruct with multi-token prediction at 66.0 tok/s using 21.9 GB on an R9700 — 2.07x the same model without MTP. Chat is barely throughput-sensitive; an agent loop is throughput-bound in a way chat is not, because every step regenerates tokens: tool calls, edits, and the reasoning between them. A 2.07x multiplier lands on every step of the loop, not just the first. Methodology and numbers are in our MTP article.
The config we actually run. Our coding-agent experience article runs Qwen3-Coder-30B at Q4_K_M inside Continue and Aider. That is not a benchmark result — it is a working setup, chosen because the measured 18.1 GB footprint and 38.6 tok/s make it practical on a single high-VRAM card. What held up and what did not is in our private coding agent write-up.
Choosing a model by VRAM tier, for agent work
Our model roundup tiers the current roster: Qwen3-Coder-30B-A3B for coding, Qwen3.6-27B for throughput, Ornith-1.0-35B for long context, LFM2.5-8B-A1B for budget builds, and Llama-3.1-8B as the dense baseline. Agent work reshuffles those tiers, because a loop stresses context and throughput before it stresses one-shot quality.
One honesty note before the table: our “survives the loop” judgments come from the coding-agent experience article — hands-on time with Continue and Aider — not from a dedicated agentic benchmark, because we have not run one yet. Treat the table as engineering guidance grounded in measured single-shot data plus field experience, not as agent-loop scores.
| VRAM tier | Model class | Expected agent workload |
|---|---|---|
| Budget | LFM2.5-8B-A1B, or Llama-3.1-8B as the dense baseline | Short tool loops, single-file edits, heavy human review; expect more failed recoveries as steps pile up |
| ~18 GB measured | Qwen3-Coder-30B-A3B at Q4_K_M (0.81 pass@1, 38.6 tok/s) | Full Continue/Aider agent sessions; the config our coding-agent article runs |
| ~22 GB measured | Qwen3.6-27B-A3B-Instruct with MTP (66.0 tok/s at 21.9 GB) | Step-heavy loops where wall-clock per step dominates; long tool outputs |
| Long-context-first | Ornith-1.0-35B | Sessions where accumulated context, not step count, is the binding constraint |
| ~40 GB (dual-GPU class) | Larger quantized coders with context headroom | The corpus reader’s scenario: long agent sessions with room for context growth |
Two rows deserve comment. The budget row is where the folklore bites: an 8B-class model can look acceptable in chat and still struggle with tool-call discipline across a long loop. Our editorial recommendation is to start agent experiments at the Qwen3-Coder-30B tier and treat budget models as completion engines rather than loop drivers — a judgment, flagged as one.
The 40 GB row is the corpus question verbatim. You do not need 40 GB to run the measured coding leader: 18.1 GB at Q4_K_M fits a single high-VRAM card. What 40 GB buys is headroom — longer contexts, larger models, or both — and it usually means a second card. Our single-vs-dual-GPU measurements show what that second card actually adds, and our VRAM planning guide walks the sizing question end to end.
The tooling layer: harnesses and guardrails
The corpus question “What are the agentic softwares you run on top of local llm which can do tasks for you?” has a short editorial answer: the harness matters as much as the model. Continue and Aider are where our own hands-on experience lives; pi.dev-style harnesses are what the beginner thread pairs with Qwen3.6-27B. We do not benchmark harnesses, so treat harness choice as field advice, not a measured ranking.
Guardrails are the other half of the loop. The 687-point Hacker News discussion on agent guardrails exists because a model in a loop takes actions, and actions need containment: sandboxed execution, diff review before writes, limited tool scopes, hard stops on destructive commands. Local changes the calculus in one useful way — you are not protecting a cloud account or an API key, you are protecting your own filesystem, and you control the harness completely. That control is the quiet case for local agentic coding even while local models trail hosted ones in loop reliability, a gap we can describe from experience but not yet quantify.
Honest caveats
- No dedicated agentic-loop benchmark exists in our lab yet. bm-003 measures single-shot HumanEval pass@1. It does not measure multi-step task completion, tool-call validity rates, or error-recovery success. When we say a model “survives the loop,” we are extrapolating from measured coding quality, measured throughput and memory, and hands-on harness experience. That is inference stacked on measurement, and we label it as such.
- The quantization delta inside loops is unmeasured. 0.86 to 0.81 is a single-shot delta. Whether Q4_K_M costs more or less than five points of agent-task success is unknown.
- Harness effects are unmeasured. The same model can behave differently in Continue than in Aider; we have not controlled for that.
- Corpus threads are demand evidence, not test results. The 40 GB question and the pi.dev pairing tell us what people are attempting, not what works.
FAQ
What is the best local model for agentic coding right now? Qwen3-Coder-30B-A3B is our measured coding leader — 0.86 HumanEval pass@1 at Q8_0, 0.81 at Q4_K_M — and it is the model our coding-agent experience article runs at Q4_K_M in Continue and Aider. The caveat stands: “best in agent loops” is a judgment from experience, not a dedicated benchmark, because we have not published one yet.
How much VRAM do I need for agentic coding? Less than the folklore suggests. Our measured anchors are 18.1 GB for Qwen3-Coder-30B-A3B at Q4_K_M and 21.9 GB for Qwen3.6-27B-A3B-Instruct with MTP. Agent loops add context on top, and the KV cache grows with context no matter how far you quantize the weights. Our VRAM planning guide covers the sizing.
Is Qwen3.6-27B plus pi.dev a good beginner starting point? It is a reasonable pairing, and the corpus thread asking about it is evidence the combination is on people’s minds. Qwen3.6-27B is our measured throughput anchor at 66.0 tok/s with MTP. But the configuration with the most direct hands-on evidence behind it on this site is Qwen3-Coder-30B at Q4_K_M in Continue/Aider. If you want a tested path rather than an assembled one, start there.
Why does my model do fine in chat but fail in agent loops? Loops stress three things chat does not: parseable tool-call output on every step, a context window filling with the model’s own tool outputs, and error recovery after its own failures. Most benchmarks, including HumanEval, are chat-shaped and do not test any of this. We cover the long-session version of this problem in our article on why local models can feel dumber than their benchmarks.
Do I need 40 GB of VRAM for agentic coding? No. The measured coding leader runs at 18.1 GB at Q4_K_M. Forty gigabytes buys headroom for longer contexts and larger models, typically via a second card; our single-vs-dual-GPU article measures what that second card actually adds.
Does quantization hurt agent behavior? The only quantization delta we have measured is single-shot: 0.86 pass@1 at Q8_0 versus 0.81 at Q4_K_M for Qwen3-Coder-30B-A3B. No loop-specific delta exists in our data yet. Our quantization guide explains the mechanics, including why the KV cache — the part that grows in agent sessions — does not shrink with the weights.