Qwen3.6-27B cross-machine baseline: dual 12GB vs single 20GB vs dual 32GB — and the RQ-001 answer
First cross-machine baseline of the lab. The same model — Qwen3.6-27B at Q4_K_M, identical Ollama digest a50eda8ed977 (byte-identical weights) — run on all four lab machines under controlled conditions (same 37-token prompt, 512 generated tokens, temperature 0, seed 42, one warmup rep). Result: the three CUDA/ROCm machines cluster within ~10% for generation (Ray 26.74, Evo-X2 25.21, Victor 24.28 tok/s), so for a 27B Q4_K_M model dual-12GB NVIDIA is NOT meaningfully faster than single-20GB AMD. The dual-GPU advantage only matters once the model exceeds ~18GB. Jitori's 8.41 tok/s is Vulkan-fallback performance, NOT the Arc B60's real capability (SYCL is broken on Battlemage — see bm-012). This directly answers Research Question RQ-001.
Configuration
- Model
- Qwen3.6 27B (qwen3.6:27b)
- Artifact
- qwen3.6:27b — GGUF Q4_K_M, Ollama digest a50eda8ed977 (identical across MSI-Command, Evo-X2, Ray; same tag pulled on Jitori)
- Checksum
- pending
- Quantization
- q4-k-m
- Context length
- 2,048 tokens
- Backend
- rocm
- Runtime
- Stock Ollama per host: CUDA 13.0 on Victor (MSI-Command), ROCm 6.17 on Evo-X2, ROCm 7.0 on Ray, Vulkan (Mesa ANV) on Jitori. Ollama versions 0.30.8–0.31.1 across the fleet.
- Settings
- Identical benchmark harness (Shared/Scripts/Benchmarking/ollama_bench.py, pure stdlib) on every host. Prompt: 'Write a 300-word essay explaining why local AI inference matters for privacy, latency, and cost. Be specific and technical.' (37 prompt tokens). Generation: 512 tokens, temperature=0, seed=42 (deterministic). One warmup prompt per run (discarded). Victor uses CUDA_VISIBLE_DEVICES=0,1 tensor-split across both 12GB GPUs. Ray/Evo-X2 use ROCm defaults. Jitori uses OLLAMA_VULKAN_DEVICE=1 (Arc B60) with OLLAMA_VRAM_BUDGET=22000. VRAM sampled via nvidia-smi / rocm-smi / Ollama /api/ps fallback (Jitori).
- Workload pack
- cross-machine-single-prompt-v1
- Author
- Edgar
- Reviewer
- Edgar
Results
Single-prompt speed + memory probe, NOT the site's 100-prompt workload-pack-v1. One fixed 37-token prompt, 512 generated tokens, temperature 0, seed 42, one discarded warmup rep, on each of 4 machines using byte-identical model weights (Ollama digest a50eda8ed977). Purpose is cross-MACHINE comparability (same harness, same weights, same prompt on every host), not absolute throughput characterization. Do NOT cross-compare these tok/s figures against workload-pack-v1 records — different protocol.
26.7tok/s
GEN · Ray
217tok/s
PROMPT
0ms
TTFT
21.2GB
VRAM USED
| MACHINE | PROMPT tok/s | GEN tok/s | TTFT ms | VRAM GB | RAM GB | POWER W |
|---|---|---|---|---|---|---|
Ray Rank 1. 2x AMD Radeon AI PRO R9700 32GB (ROCm 7.0, gfx1201 Navi 48). Generation 26.735 tok/s, prompt-eval 217.27 tok/s, end-to-end wall 19.71s. Peak VRAM 21753 MiB = 21.24 GB (34% of 64GB combined). RAM delta 412 MiB (model fully in VRAM). Backend: ROCm. | 217.3 | 26.7 | 0 | 21.2 | 0.4 | 0 |
EvoX2 Rank 2. AMD Radeon RX 7900 XT 20GB (ROCm 6.17, gfx1100). Generation 25.214 tok/s, prompt-eval 162.05 tok/s, end-to-end wall 20.84s. Peak VRAM 17222 MiB = 16.82 GB (86% of 20GB — tightest fit of the four). RAM delta 282 MiB. Backend: ROCm. The iGPU (Radeon 8060S, gfx1151) was NOT used for this run — discrete RX 7900 XT only. | 162.1 | 25.2 | 0 | 16.8 | 0.3 | 0 |
Victor (msi-command) Rank 3 (generation) but Rank 1 (prompt-eval by ~2x). MSI Vector 16 HX: 2x NVIDIA RTX 5070 12GB (CUDA 13.0, Blackwell sm_120), tensor-split across both via CUDA_VISIBLE_DEVICES=0,1. Generation 24.279 tok/s, prompt-eval 344.85 tok/s (highest of the four — Blackwell batched prefill advantage), end-to-end wall 21.57s. Peak VRAM 9257 MiB = 9.04 GB (39% of 24GB combined; model fits in ONE 12GB card). RAM delta ~0 MiB. Backend: CUDA. This is the RQ-001 dual-12GB data point. | 344.9 | 24.3 | 0 | 9.0 | 0.0 | 0 |
Jitori PC Rank 4 — CAVEAT: this is Vulkan-FALLBACK performance, NOT the Arc B60's real capability. Intel Arc Pro B60 24GB (Battlemage BMG G21). Stock Ollama on this host has no SYCL runner, so it falls back to the generic Vulkan shim. Generation 8.405 tok/s, prompt-eval 111.94 tok/s, end-to-end wall 61.72s. Peak VRAM 19722 MiB = 19.26 GB (82% of 24GB). RAM delta 350 MiB. A SYCL rebuild attempt (llama.cpp build-sycl-nograph, documented Battlemage env vars) produced 5.97 tok/s with garbage output — matches upstream bug ggml-org/llama.cpp#22413. The real Arc B60 number requires either an older known-good oneAPI tag or the OpenVINO path (see bm-012). Backend: Vulkan (fallback). | 111.9 | 8.4 | 0 | 19.3 | 0.3 | 0 |
Limitations
- ◆WORKLOAD MISMATCH: this is a single-prompt probe (cross-machine-single-prompt-v1), NOT the site's 100-prompt workload-pack-v1. One prompt, 512 tokens, one warmup rep. The decision value is the cross-MACHINE comparison (identical weights + harness on every host), not an absolute throughput claim. Do not rank these tok/s numbers against workload-pack-v1 records.
- ◆BACKEND LABEL: this is a multi-backend cross-vendor record (CUDA on Victor, ROCm on Evo-X2 + Ray, Vulkan on Jitori). The schema's single backend field is set to 'rocm' (the rank-1 machine's backend); each result row's notes state that machine's actual backend. There is no 'multi' enum value.
- ◆TTFT NOT MEASURED: the harness recorded end-to-end wall time (prompt eval + generation), not time-to-first-token. ttft_ms is set to 0 (the documented 'not measured' sentinel) on every row. End-to-end wall time IS reported in each row's notes. A derived TTFT approximation (prompt_tokens / prompt_eval_tok_sec) exists but is not published as a measured field.
- ◆POWER NOT MEASURED: power_watts is 0 on every row (not captured by this harness revision).
- ◆JITORI IS FALLBACK PERFORMANCE: the 8.41 tok/s is Vulkan-fallback, not the Arc B60's real capability. Stock Ollama has no SYCL runner on Battlemage; the SYCL rebuild attempt produced garbage output (upstream bug llama.cpp#22413). Treat Jitori's row as 'Vulkan shim on Arc B60', not 'Arc B60 inference speed'. See bm-012 for the OpenVINO path that DOES work on this GPU.
- ◆RQ-001 IS NOT A CLEAN VENDOR COMPARISON: Victor runs CUDA while Evo-X2/Ray run ROCm, so the dual-12GB (Victor) vs single-20GB (Evo-X2) comparison conflates GPU topology AND vendor backend. The ~10% generation spread is within vendor-backend variance. A clean topology test would need dual-vs-single WITHIN the same vendor, which the current fleet cannot do.
- ◆SINGLE REP PER MACHINE: one measured generation per machine (after one warmup). Variance characterization would need multiple reps; the deterministic seed + temp 0 keep output stable, but tok/s can vary run-to-run by a few percent. The cross-machine ranking (Ray > Evo-X2 > Victor) is within noise for the top 3 — treat them as a cluster, not a strict ranking.
- ◆VRAM UNIT: peak_vram_mib from the harness converted to GB via /1024 (binary). Victor's 9257 MiB = 9.04 GB; the model (17GB Q4_K_M) appears to use less because Ollama reports active allocation, not the full mmap footprint.
Failures & dead ends
Corrections
2026-07-19
Initial publication. First cross-machine baseline of the lab. Answers RQ-001 (dual-12GB vs single-24GB) for the 27B Q4_K_M model class: at this size, dual-12GB NVIDIA is not meaningfully faster than single-20GB AMD — the three CUDA/ROCm machines cluster within ~10%. Jitori's Vulkan-fallback number is explicitly flagged as not representative of the Arc B60 (SYCL regression, see bm-012 for the working OpenVINO path).