Ollama vs llama.cpp on the same hardware: measured

"Ollama is easier, llama.cpp is faster." That framing is from 2024, and it was correct then. What it never came with — in any Reddit thread, any Medium post, any YouTube comparison — was the actual measured gap on the same card with the same weights. The numbers float around unmoored: "Ollama is slower," "Ollama is fine for chat," "llama.cpp is 20% faster," "llama.cpp is 2× faster." None of those claims names the card, the model, the quant, or the flags.

This is that gap, measured. Four backend configurations on a single Radeon AI PRO R9700 (gfx1201 RDNA4), same Qwen3.6-27B-A3B Q4_K_M-mtp weights, back-to-back. The headline finding is uncomfortable for both sides of the framing: the gap is real but smaller than the folklore (1.2× with matched features), and the dramatic blowouts come from features Ollama doesn't expose, not from Ollama being a different engine.

For the runtime-layer framing — when to pick Ollama vs llama.cpp vs vLLM vs MLX conceptually — the parent piece is Ollama vs llama.cpp in 2026. This article is the measured same-hardware data that one explicitly deferred.

What "same hardware" actually means

Most "Ollama vs llama.cpp" comparisons change two or three variables at once: different card, different model, different quant, different prompt. That makes the gap unattributable. The four records in our library that hold those variables constant:

• bm-004 — Qwen3.6-27B-A3B Q4_K_M-mtp on the same single R9700, four backends back-to-back
• bm-005 — Ornith-1.0-35B Q4_K_M on the same single R9700, Ollama vs llama.cpp-Vulkan
• bm-013 — Qwen3.6-27B Q4_K_M across four machines, stock Ollama, identical harness
• bm-012 — Qwen3-30B-A3B INT4 on the Arc B60, OVMS vs llama.cpp-Vulkan (not Ollama, but the same runtime-vs-runtime question on Intel)

Same machine, same sha256-verified weights, same prompt shape. That is the bar. Anything below it is a chart fight.

Measured: Qwen3.6-27B on the R9700

bm-004 is the cleanest record in the library for this question. Same card, same model, four backend configurations:

Backend (same R9700, same Qwen3.6-27B-mtp)	Generation tok/s	VRAM
llama.cpp Vulkan + MTP speculative decoding	66.0	21.9 GB
llama.cpp Vulkan, no MTP	31.9	18.8 GB
llama.cpp HIP (ROCm-style)	27.1	19.1 GB
Ollama (HIP)	26.7	22.3 GB

Three findings.

First, with matched features, llama.cpp Vulkan beats Ollama by ~1.2× (31.9 vs 26.7 tok/s). That is a real gap but a small one — not the "2× faster" folklore. For single-user chat at 27 tok/s, both are above reading speed.

Second, llama.cpp's Vulkan slightly outperforms its HIP on this RDNA4 card (31.9 vs 27.1, ~1.18×). That is not supposed to happen — HIP is the vendor backend — and it's a signal that ROCm maturity on RDNA4 was still catching up as of this build. See the backend comparison for the fuller picture.

Third, the dramatic gap comes from MTP speculative decoding, not from "llama.cpp is faster". The Vulkan+MTP path more than doubles throughput (66 vs 27 tok/s). That's not a property of llama.cpp the engine — it's a property of a feature Ollama's runner didn't activate as of this build. The full deep-dive on what MTP is and why it works is in Speculative decoding on consumer GPUs.

Measured: Ornith-35B on the R9700

bm-005 is the result that gets quoted out of context. Same single R9700, Ornith-1.0-35B Q4_K_M:

Backend (same R9700, same Ornith-35B)	Generation tok/s	VRAM
Ollama (HIP)	78.3	23.2 GB
llama.cpp Vulkan	27.3	34.2 GB

Read naively, Ollama is 2.9× faster than llama.cpp here — the opposite of the folklore. That read is wrong. The VRAM column is why: llama.cpp-Vulkan spilled past the card's 32GB into system RAM (34.2GB), spending most of its time paging weight tensors across the PCIe bus instead of doing math. Ollama's runner chose a configuration that fit in 23.2GB. Ollama "won" because it stayed in VRAM, not because its inference was faster at the same workload.

This is the kind of result that演示 why same-card isn't enough — same-card-and-same-VRAM-footprint is the real bar. The lesson for benchmark readers: any tok/s comparison that doesn't show VRAM used is hiding the most important variable.

Measured: stock Ollama across the fleet

bm-013 ran stock Ollama — no tuning, default flags, the path of least resistance — across all four machines in our fleet on identical Qwen3.6-27B Q4_K_M weights. This is the "what you actually get the moment you install Ollama and pull the model" data point.

Machine	Ollama backend	Generation tok/s
ray (2× R9700, ROCm 7.0)	HIP	26.74
evox2 (RX 7900 XT, ROCm 6.17)	HIP	25.21
victor (2× RTX 5070, CUDA 13.0)	CUDA	24.28
jitori (Arc B60, Vulkan fallback)	Vulkan	8.41 ⚠

The three CUDA/ROCm machines cluster within ~10% on generation. The jitori row is the explicit caveat: stock Ollama has no SYCL runner for Battlemage, so it falls back to Vulkan and gets 8.41 tok/s — not the Arc B60's real capability. See the backend piece for what OVMS delivers on the same card (67.95 tok/s).

Why the gap exists

Ollama is not a different engine. It wraps llama.cpp — the GGML kernels, the quantization formats, the GPU dispatch are all the same code. The overhead comes from three places:

1. Runner selection. Ollama picks a runner binary per model based on its detected capabilities. That choice is sometimes conservative — picking a slower runner because the faster one had a bug two releases ago. The runner cache lags llama.cpp main.
2. Default flags. Ollama's defaults are tuned for "works everywhere," not "fastest on your hardware." KV-cache size, batch parameters, and thread counts are mid-range compromises. llama.cpp with -t 8 -b 512 -ngl 99 tuned to your card will beat Ollama's defaults on the same workload.
3. Feature lag. Anything new in llama.cpp — MTP, experimental attention backends, custom offload strategies — lands in Ollama only after Ollama's release cycle picks it up and exposes it. MTP on Qwen3.6-27B is the example in our data.

The practical implication: the gap shrinks when you tune llama.cpp down to Ollama's defaults, and grows when you tune llama.cpp up. It is not a property of "Ollama is slower" — it is a property of "Ollama's defaults are conservative and llama.cpp's ceiling is higher."

The honest caveats

What this article did NOT measure:

• vLLM and SGLang. Those are serving runtimes for concurrent multi-user throughput. Every number here is single-stream or single-prompt. The "Ollama vs llama.cpp" question for desktop single-user chat is what we measured; the "Ollama vs vLLM" question for serving is different and we have not run it. The parent piece (Ollama vs llama.cpp in 2026) covers the conceptual serving-tier story with cited third-party data.
• Concurrent requests. Every result above is one prompt at a time. The gap widens under concurrency, where llama.cpp's manual batching and vLLM's continuous batching both beat Ollama's sequential behavior.
• MTP generality. The 66 tok/s blowout requires the model to ship an MTP draft head. Qwen3.6-27B does; most models on Ollama's library don't. Don't generalize the MTP number to every workload.
• Ollama moves fast. The HIP runner not activating MTP, the conservative defaults, the runner-lag — all of these may narrow in future Ollama releases. Check the current release notes before treating any specific number above as permanent.
• Single-prompt workloads in bm-004/005. The 27B/35B records use single-prompt runs, not the workload-pack-v1 suite that bm-001/002/003 use. Don't cross-compare.

The decision, by use case

You want...	Use
Easiest desktop chat, single user, model fits comfortably	Ollama — the 1.2× gap doesn't justify the setup.
Maximum single-stream speed, willing to build/tune	llama.cpp — tune flags per card, get the 1.2–2× over Ollama.
MTP speculative decoding (model supports it)	llama.cpp — Ollama's runner didn't activate MTP as of our build (bm-004).
Multi-vendor benchmarking, Vulkan-on-NVIDIA experiments	llama.cpp — full backend control. See the backend piece.
Concurrent multi-user serving	Neither — vLLM or SGLang (or OVMS on Arc). See the runtime parent piece.
Just-installed-Ollama baseline	Expect the bm-013 numbers — usable, not optimal.

The takeaway

The "Ollama is easier, llama.cpp is faster" framing is correct but underspecified. With matched features, llama.cpp beats Ollama by ~1.2× on a 27B model — real but small. The dramatic blowouts come from features Ollama doesn't expose (MTP), from tuning llama.cpp above Ollama's conservative defaults, or from confounded comparisons where one side spilled to system RAM. Pick Ollama for chat, llama.cpp for measured speed, and vLLM/SGLang for serving — and never compare two runtimes without naming the card, the weights, the VRAM footprint, and the flags.

The runtime-layer framing — when to pick which tool conceptually — is in Ollama vs llama.cpp in 2026. The layer underneath — the compute backend each runtime dispatches to — is in CUDA vs ROCm vs Vulkan vs OpenVINO. The raw records behind every number above are in the benchmark library.

Last verified: July 2026. Ollama and llama.cpp both move fast — re-validate against current release notes before relying on any specific tok/s number above.