Qwen3.6-27B + MTP backend showdown on Radeon AI PRO R9700
On a single Radeon AI PRO R9700 (gfx1201, RDNA4), llama.cpp Vulkan with the MTP speculative-decoding head is the only path that activates Qwen3.6's Multi-Token Prediction — delivering 66 tok/s, roughly 2.5x the 27 tok/s class of every other backend. If you don't load the -mtp.gguf draft head, Vulkan, HIP, and Ollama all land within a narrow 26-32 tok/s band and the architecture's main speed advantage is left on the table.
Configuration
- Model
- Qwen3.6-27B-A3B-Instruct (with MTP draft head)
- Artifact
- Qwen3.6-27B-Q4_K_M-mtp.gguf (+ base Qwen3.6-27B-Q4_K_M.gguf)
- Checksum
- pending
- Quantization
- q4-k-m
- Context length
- 65,536 tokens
- Backend
- vulkan
- Runtime
- llama.cpp (Vulkan build, b3500-era) + Ollama (HIP) for comparison
- Settings
- -ngl 999 -dev Vulkan0 -fa on -ctk f16 -ctv f16 -c 65536 -b 2048 -ub 512 --spec-type draft-mtp --spec-draft-n-max 3 --no-webui --parallel 1 (winner); see limitations for the other backends' flags
- Workload pack
- ray-single-prompt-coding-v1
- Author
- Edgar
- Reviewer
- Edgar
Results
Single fixed coding prompt ('Write a Python function that implements merge sort with type hints and docstrings'), 256-512 token generation, temperature=0, 3 runs per backend, best result reported. This is a narrower workload than the site's workload-pack-v1 (100-prompt HumanEval) and is preserved here for backend comparability, not absolute quality claims.
66.0tok/s
GEN · Ray
600tok/s
PROMPT
0ms
TTFT
21.9GB
VRAM USED
| MACHINE | PROMPT tok/s | GEN tok/s | TTFT ms | VRAM GB | RAM GB | POWER W |
|---|---|---|---|---|---|---|
Ray GPU0 (one of two R9700s). WINNER. llama.cpp Vulkan + MTP speculative decoding (--spec-type draft-mtp, --spec-draft-n-max 3). Prompt speed is approximate (~600, reported as a band by the operator). | 600.0 | 66.0 | 0 | 21.9 | 0.0 | — |
Ray GPU0. llama.cpp Vulkan, MTP head NOT loaded. Baseline speculative-decoding-off comparison. | 600.0 | 31.9 | 0 | 18.8 | 0.0 | — |
Ray GPU0. llama.cpp HIP backend. Vulkan outperforms HIP by ~1.18x on gfx1201 for this model. | 563.0 | 27.1 | 0 | 19.1 | 0.0 | — |
Ray GPU0. Ollama (uses HIP internally). Same speed class as llama.cpp HIP, as expected. | 576.0 | 26.7 | 0 | 22.3 | 0.0 | — |
Limitations
- ◆ttft_ms and ram_used_gb are recorded as 0 because the source measurement (RAY_BENCHMARK_RESULTS.md, 2026-06-30) did not capture time-to-first-token or system-RAM usage; these fields are required by the site schema and 0 is used as an explicit 'not measured' sentinel, not a real measurement.
- ◆prompt_speed_tps values (~563-600) are operator-reported approximations, not instrumented measurements; treat as bands, not precise figures.
- ◆Workload is a single coding prompt, not the site's 100-prompt HumanEval workload-pack-v1. Conclusions are about backend RELATIVE ordering on this hardware, not absolute model quality. Do not compare these tok/s numbers directly against workload-pack-v1 results.
- ◆Only the Vulkan+MTP path activates the MTP draft head; all other backends run the base model without speculative decoding, which is the point of the comparison but means the 2.5x gap is specific to MTP-capable models.
- ◆One GPU per model (no tensor splitting across the two R9700s); combined-GPU scaling is a separate benchmark.
- ◆Raw evidence link (/lab/raw/bm-004/) currently resolves to a stub; full raw output to be uploaded before or within 30 days of launch.
- ◆model_checksum not recorded at measurement time; capture on next re-run.