● CALIBRATED 2026-10-04 · REC 000
Local AI Frontier

Benchmarks / BM-005

BM-005 · TESTED 2026-06-30

Ornith-1.0-35B backend showdown on Radeon AI PRO R9700

For the Ornith-1.0-35B MoE model (qwen35moe family) on a single Radeon AI PRO R9700, Ollama is the clear winner at 78.3 tok/s - roughly 2.9x faster than llama.cpp Vulkan (27.3 tok/s), which also spilled past the 32GB VRAM limit into system RAM. The HIP backend failed outright: it could not parse the Ornith GGUF (version mismatch). On MoE models without an MTP head, Ollama's expert routing currently beats hand-tuned llama.cpp on this build.

vulkan · q4-k-m · Ollama (HIP) + llama.cpp (Vulkan) + llama.cpp (HIP, failed)

Configuration

Artifact
ornith-1.0-35b-Q4_K_M.gguf
Author
Edgar
Backend
vulkan
Checksum
Context length
96256 tokens
Model
Ornith-1.0-35B
Quantization
q4-k-m
Runtime
Ollama (HIP) + llama.cpp (Vulkan) + llama.cpp (HIP, failed)
Settings
Ollama winner: Modelfile FROM ornith-1.0-35b-Q4_K_M.gguf, num_ctx 96256, num_predict 32768, temperature 0, num_gpu 99, pinned to GPU1 via ROCR_VISIBLE_DEVICES=1, OLLAMA_KEEP_ALIVE=-1. llama.cpp Vulkan/HIP runs used comparable -ngl/-c flags; see limitations.
Workload pack
ray-single-prompt-coding-v1

Results

Single fixed coding prompt ('Write a Python function that implements merge sort with type hints and docstrings'), 256-512 token generation, temperature=0, 3 runs per backend, best result reported. Same protocol as bm-004 (run back-to-back on the same session, GPU0 vs GPU1). Narrower than the site's workload-pack-v1; preserved for backend comparability, not absolute quality claims.

MACHINEPROMPT tok/sGEN tok/sTTFT msVRAM GBRAM GBPOWER W
Ray GPU1 (one of two R9700s). WINNER. Ollama. Notable result for a 35B model; attributed to Ollama's optimized MoE expert routing for the qwen35moe family. Pinned to GPU1; persistent via systemd (OLLAMA_KEEP_ALIVE=-1).1592.078.3023.20.0—
Ray GPU1. llama.cpp Vulkan. VRAM OVERFLOWED the 32GB card and spilled to system RAM (34.2GB reported total) - this is why it is 2.9x slower, not a pure backend-quality gap. The model does not cleanly fit a single R9700 via this path.564.027.3034.20.0—

Limitations

  • ttft_ms and ram_used_gb are recorded as 0 because the source measurement (RAY_BENCHMARK_RESULTS.md, 2026-06-30) did not capture time-to-first-token or system-RAM usage; these fields are required by the site schema and 0 is used as an explicit 'not measured' sentinel, not a real measurement. (Note: the Vulkan run's 34.2GB is GPU+RAM spill combined, reported in vram_used_gb per the source's framing - see the result note.)
  • prompt_speed_tps values are operator-reported approximations, not instrumented measurements; treat as bands.
  • Workload is a single coding prompt, not the site's 100-prompt HumanEval workload-pack-v1. Conclusions are about backend RELATIVE ordering on this hardware, not absolute model quality. Do not compare these tok/s numbers directly against workload-pack-v1 results.
  • The 2.9x Ollama-vs-Vulkan gap is partly confounded by VRAM: Vulkan spilled past 32GB into system RAM while Ollama fit in 23.2GB. So this is 'Ollama's routing + fit' vs 'Vulkan's routing + spill', not a clean backend-only comparison. A fair re-test would cap context/flags so both paths fit in VRAM.
  • The HIP failure (see failures[]) is specific to this llama.cpp build's GGUF-version handling for the qwen35moe architecture; it may not reproduce on newer builds.
  • One GPU per model (no tensor splitting across the two R9700s).
  • Raw evidence link (/lab/raw/bm-005/) currently resolves to a stub; full raw output to be uploaded before or within 30 days of launch.
  • model_checksum not recorded at measurement time; capture on next re-run.

Failures & dead ends

  • llama.cpp HIP backend could not parse the Ornith-1.0-35B GGUF - version mismatch on this build (qwen35moe family). No result produced. See limitations for reproducibility caveat.

RAW DATA → How we test →

RELATED EQUIPMENT GUIDES

If these numbers point you toward hardware, these buyer-intent guides go deeper. Links below are editorial; they carry no affiliate tag.