Vulkan vs SYCL vs CUDA: the backend decision

Three compute backends matter for consumer local AI: CUDA, ROCm/Vulkan (AMD), and Vulkan/SYCL (Intel Arc). The choice is not academic — it changes your numbers and, in one case, whether your output is even correct.

The Battlemage caveat (read this first)

CUDA (NVIDIA)

The mature ecosystem. cuBLAS, FlashAttention, and the broadest runtime support. Our head-to-head shows ~35% faster generation vs Arc Vulkan at the same 24GB tier. The cost is dollars and watts — NVIDIA is pricier per GB and draws more power.

Vulkan (AMD and Intel)

The cross-platform fallback that has become the practical default for both AMD RDNA3–4 and Intel Arc. llama.cpp's Vulkan backend is consistent and surprising-fast. If you run AMD consumer hardware, start with Vulkan before fighting ROCm.

ROCm (AMD)

Theoretically the AMD-native fast path. In practice, ROCm on consumer RDNA cards is finicky — supported-SKU lists lag, and HIP dependency chains break. For llama.cpp workloads on RX 7900 / 9700, Vulkan is currently the more reliable choice. ROCm wins on specific workloads (large-batch matrix ops) but adds operational burden.

SYCL (Intel)

Intel's native path. Works on older Arc Alchemist; broken on Battlemage. Until Intel fixes the BMG G21 SYCL pipeline, treat Vulkan as the only correct choice for Arc B-series.

Decision flowchart

1. NVIDIA card → CUDA. Done.
2. AMD RDNA3–4 consumer card → Vulkan via llama.cpp. Try ROCm only if you hit a Vulkan limitation.
3. Intel Arc Alchemist → either Vulkan or SYCL.
4. Intel Arc Battlemage (B-series) → Vulkan only. SYCL is broken.