Benchmarks / BM-004
BM-004 · TESTED 2026-06-30
Qwen3.6-27B + MTP backend showdown on Radeon AI PRO R9700
On a single Radeon AI PRO R9700 (gfx1201, RDNA4), llama.cpp Vulkan with the MTP speculative-decoding head is the only path that activates Qwen3.6's Multi-Token Prediction — delivering 66 tok/s, roughly 2.5x the 27 tok/s class of every other backend. If you don't load the -mtp.gguf draft head, Vulkan, HIP, and Ollama all land within a narrow 26-32 tok/s band and the architecture's main speed advantage is left on the table.
vulkan · q4-k-m · llama.cpp (Vulkan build, b3500-era) + Ollama (HIP) for comparison
Configuration
- Artifact
- Qwen3.6-27B-Q4_K_M-mtp.gguf (+ base Qwen3.6-27B-Q4_K_M.gguf)
- Author
- Edgar
- Backend
- vulkan
- Checksum
- Context length
- 65536 tokens
- Model
- Qwen3.6-27B-A3B-Instruct (with MTP draft head)
- Quantization
- q4-k-m
- Runtime
- llama.cpp (Vulkan build, b3500-era) + Ollama (HIP) for comparison
- Settings
- -ngl 999 -dev Vulkan0 -fa on -ctk f16 -ctv f16 -c 65536 -b 2048 -ub 512 --spec-type draft-mtp --spec-draft-n-max 3 --no-webui --parallel 1 (winner); see limitations for the other backends' flags
- Workload pack
- ray-single-prompt-coding-v1
Results
Single fixed coding prompt ('Write a Python function that implements merge sort with type hints and docstrings'), 256-512 token generation, temperature=0, 3 runs per backend, best result reported. This is a narrower workload than the site's workload-pack-v1 (100-prompt HumanEval) and is preserved here for backend comparability, not absolute quality claims.
| MACHINE | PROMPT tok/s | GEN tok/s | TTFT ms | VRAM GB | RAM GB | POWER W |
|---|---|---|---|---|---|---|
| Ray GPU0 (one of two R9700s). WINNER. llama.cpp Vulkan + MTP speculative decoding (--spec-type draft-mtp, --spec-draft-n-max 3). Prompt speed is approximate (~600, reported as a band by the operator). | 600.0 | 66.0 | 0 | 21.9 | 0.0 | — |
| Ray GPU0. llama.cpp Vulkan, MTP head NOT loaded. Baseline speculative-decoding-off comparison. | 600.0 | 31.9 | 0 | 18.8 | 0.0 | — |
| Ray GPU0. llama.cpp HIP backend. Vulkan outperforms HIP by ~1.18x on gfx1201 for this model. | 563.0 | 27.1 | 0 | 19.1 | 0.0 | — |
| Ray GPU0. Ollama (uses HIP internally). Same speed class as llama.cpp HIP, as expected. | 576.0 | 26.7 | 0 | 22.3 | 0.0 | — |
Limitations
- ttft_ms and ram_used_gb are recorded as 0 because the source measurement (RAY_BENCHMARK_RESULTS.md, 2026-06-30) did not capture time-to-first-token or system-RAM usage; these fields are required by the site schema and 0 is used as an explicit 'not measured' sentinel, not a real measurement.
- prompt_speed_tps values (~563-600) are operator-reported approximations, not instrumented measurements; treat as bands, not precise figures.
- Workload is a single coding prompt, not the site's 100-prompt HumanEval workload-pack-v1. Conclusions are about backend RELATIVE ordering on this hardware, not absolute model quality. Do not compare these tok/s numbers directly against workload-pack-v1 results.
- Only the Vulkan+MTP path activates the MTP draft head; all other backends run the base model without speculative decoding, which is the point of the comparison but means the 2.5x gap is specific to MTP-capable models.
- One GPU per model (no tensor splitting across the two R9700s); combined-GPU scaling is a separate benchmark.
- Raw evidence link (/lab/raw/bm-004/) currently resolves to a stub; full raw output to be uploaded before or within 30 days of launch.
- model_checksum not recorded at measurement time; capture on next re-run.
RELATED EQUIPMENT GUIDES
If these numbers point you toward hardware, these buyer-intent guides go deeper. Links below are editorial; they carry no affiliate tag.
-
The measured GPU buyer's guide
Turn this result into a hardware decision.
-
How much VRAM you actually need
Will your target model fit?
-
Build a rig around these numbers
A complete sub-$1,000 local-AI lab.