Llama-3.1-8B-Instruct dense 8B on Arc B60 vs RX 7900 XT — general-work cross-GPU
Llama-3.1-8B-Instruct (Q4_K_M, 8.03B dense params, 4.58 GiB) measured on TWO GPUs for a clean dense-vs-GPU comparison: Jitori's Intel Arc Pro B60 (Battlemage, Vulkan) at 27.01 tok/s gen / 706.61 tok/s prompt, and EvoX2's AMD Radeon RX 7900 XT (gfx1100, ROCm) at 105.22 tok/s gen / 3210.77 tok/s prompt. The RX 7900 XT is ~3.9x faster on this dense 8B than the Arc B60 - the SAME ratio as on the MoE LFM2.5-8B (see bm-007), confirming the GPU gap is consistent across architectures. Closes the general-work workload and dense-general model-class gaps; the 9th of 10.
Configuration
- Model
- Llama-3.1-8B-Instruct
- Artifact
- Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf
- Checksum
- sha256:667b0c1932bc6ffc593ed1d03f895bf2dc8dc6df21db3042284a6f4416b06a29
- Quantization
- q4-k-m
- Context length
- 2,048 tokens
- Backend
- vulkan
- Runtime
- Arc B60: llama.cpp build be4a6a63e (Vulkan, Mesa ANV). RX 7900 XT: llama.cpp build c57607016 (ROCm 7.2.2 HIP). Same artifact (sha256-matched) on both.
- Settings
- -p 512 -n 256 -ngl 99 -r 3 (llama-bench, 3 reps) on both. Arc B60: GGML_VK_VISIBLE_DEVICES=1 (isolates Arc, hides AMD 780M iGPU per fleet doc). RX 7900 XT: HIP_VISIBLE_DEVICES=0 (isolates the discrete dGPU). Same flags, same artifact, same llama-bench protocol - only the GPU differs.
- Workload pack
- llama-bench-pp-tg-v1
- Author
- Edgar
- Reviewer
- Edgar
Results
llama-bench standard prompt-processing (pp512) and generation (tg256) tests, 3 runs each, best-of-3 with variance reported. Run on TWO GPUs (Arc B60 Vulkan, RX 7900 XT ROCm) with the SAME artifact and SAME flags - this is a controlled cross-GPU comparison, not just a single-platform speed test. This is NOT the site's workload-pack-v1 (a general-instruct prompt suite with reasoning/writing/Q&A rubric) - it is the canonical llama.cpp throughput benchmark. The 'general-work' coverage is closed by the *choice of model* (Llama-3.1-8B-Instruct is a dense general-purpose assistant, not a coder specialist or MoE); a true general-workload quality suite (MMLU-style + open-ended writing) is a documented follow-up. The publishable insight is the 2x2 with bm-007: see limitations.
105.2tok/s
GEN · EvoX2
3211tok/s
PROMPT
0ms
TTFT
4.6GB
VRAM USED
| MACHINE | PROMPT tok/s | GEN tok/s | TTFT ms | VRAM GB | RAM GB | POWER W |
|---|---|---|---|---|---|---|
Jitori PC Jitori Intel Arc Pro B60 (Battlemage BMG G21, 24GB VRAM) via Mesa Vulkan. GGML_VK_VISIBLE_DEVICES=1 isolated the Arc (device: 'Intel(R) Arc(tm) Pro B60 Graphics (BMG G21)', uma: 0, fp16: 1, warp size: 32, matrix cores: KHR_coopmat). pp512 = 706.61 ± 1.78 t/s, tg256 = 27.01 ± 0.00 t/s (3 runs each; generation variance was zero). Build be4a6a63e (9775). vram_used_gb recorded as 4.58 (model footprint) - Battlemage has no clean per-process VRAM counter via sysfs; see limitations. | 706.6 | 27.0 | 0 | 4.6 | 0.0 | 0 |
EvoX2 EvoX2 AMD Radeon RX 7900 XT (gfx1100, 20GB VRAM) via ROCm 7.2.2. HIP_VISIBLE_DEVICES=0 isolated the discrete dGPU. pp512 = 3210.77 ± 47.45 t/s, tg256 = 105.22 ± 0.03 t/s (3 runs each, llama-bench). Build c57607016 (9743). Same artifact (sha256 667b...06a29) as the Arc run - this is an apples-to-apples GPU comparison. vram_used_gb recorded as 4.58 (model footprint); rocm-smi device-index reporting was inconsistent under HIP_VISIBLE_DEVICES filtering on this dual-GPU box, so the clean runtime number wasn't captured. Model fits trivially in 20GB. | 3210.8 | 105.2 | 0 | 4.6 | 0.0 | 0 |
Limitations
- ◆vram_used_gb is recorded as 4.58 (the model's on-disk Q4_K_M footprint) on both rows, not measured runtime allocations. Arc B60 (Battlemage): no clean per-process VRAM counter via sysfs (no i915 memory node resolved); intel_gpu_top needs an interactive TTY. RX 7900 XT: rocm-smi device-index reporting was inconsistent under HIP_VISIBLE_DEVICES filtering on this dual-GPU box (readings flipped between device 0 and device 1 across calls), so a trustworthy per-process number wasn't captured. The 4.58 GiB model + KV cache fits trivially in both 20GB and 24GB cards (headroom 15-19GB), so VRAM was not a constraint - throughput is the decision-relevant number. Same pattern as bm-009.
- ◆ttft_ms, ram_used_gb, and power_watts are 0 because llama-bench does not report time-to-first-token, system RAM, or sustained power.
- ◆Workload is llama-bench pp512+tg256, NOT the site's workload-pack-v1. context_length is 2048 (the default for both builds; neither supports -c). Llama-3.1-8B's native context is 128K - throughput reflects short-context inference only.
- ◆Backends differ between rows by design: Arc B60 uses Vulkan (the path we tested with llama.cpp); RX 7900 XT uses ROCm/HIP (mature on gfx1100). Vulkan is NOT the only working Battlemage path - OpenVINO Model Server works on this same B60 (see bm-012, ~1.76x faster than Vulkan on a 30B-A3B) - we simply tested Vulkan first. This is NOT a pure GPU-vs-GPU comparison - it's GPU+backend-vs-GPU+backend. The 3.9x ratio reflects what each platform delivers with the backend we benchmarked, not a silicon-only comparison.
- ◆OpenCL noise on the Arc build: build be4a6a63e compiled Vulkan+OpenCL; OpenCL dropped the Arc as 'unsupported' (missing Adreno kernels), so Vulkan did all compute. The 'Vulkan,OpenCL' backend label is cosmetic.
- ◆model_checksum captured (sha256:667b...06a29) from the Ollama blob identifier - same hash on both machines confirms byte-identical artifact.
Corrections
2026-07-18
Extended from single-GPU (Arc B60 only) to a 2-GPU comparison by adding the RX 7900 XT row. Same artifact (sha256-matched), same flags. Corrects the original publication's framing: the 'dense-vs-MoE ~10x speed gap' insight in the v1 summary was confounded (it compared Llama-on-Arc to LFM-on-RX, mixing architecture with GPU). The clean architecture effect at matched GPU is ~2.5x (see bm-007 for the companion LFM rows), and the GPU effect is ~3.9x consistently across both architectures. v1 numbers unchanged; framing corrected.