Qwen3-Coder-30B-A3B on Victor (RTX 5070 dual-GPU, CUDA) — bm-002 re-measurement
Re-measurement of the NVIDIA CUDA path that bm-002 reported provisionally. On Victor's dual RTX 5070 mobile config (Blackwell sm_120, CUDA 13.3), Qwen3-Coder-30B-A3B at Q4_K_M generates at 172 tok/s and processes prompts at 3403 tok/s — roughly 3.3x and 5.6x faster than bm-002's provisional figures (52.1 / 612.4). bm-002 is superseded; these are the real numbers.
Configuration
- Model
- Qwen3-Coder-30B-A3B-Instruct
- Artifact
- qwen3-coder-30b-q4_k_m.gguf
- Checksum
- sha256:pending-capture-on-next-load
- Quantization
- q4-k-m
- Context length
- 8,192 tokens
- Backend
- cuda
- Runtime
- llama.cpp build 8452824 (CUDA 13.3, Blackwell sm_120)
- Settings
- -ngl 99 -p 512 -n 256 -r 3 (llama-bench); model auto-splits across both 5070s. Server-mode run used -c 8192 for VRAM/power capture.
- Workload pack
- llama-bench-pp-tg-v1
- Author
- Edgar
- Reviewer
- Edgar
Results
llama-bench standard prompt-processing (pp512) and generation (tg256) tests, 3 runs each, best-of-3 with variance reported. This is NOT the site's workload-pack-v1 (100-prompt HumanEval) - it is the canonical llama.cpp throughput benchmark, useful for direct cross-platform speed comparison but not for quality claims. Quality score carried over from bm-002 unchanged (same model artifact); a real HumanEval re-run is a follow-up.
172.1tok/s
GEN · Victor (msi-command)
3403tok/s
PROMPT
0ms
TTFT
19.0GB
VRAM USED
| MACHINE | PROMPT tok/s | GEN tok/s | TTFT ms | VRAM GB | RAM GB | POWER W |
|---|---|---|---|---|---|---|
Victor (msi-command) Victor (msi-command, MSI Vector 16 HX laptop). Dual-GPU tensor split: RTX 5070 (9.72GB) + RTX 5070 Ti Laptop (9.32GB) = ~19.0GB at -c 8192. pp512 = 3403.49 ± 42.65 t/s, tg256 = 172.05 ± 3.64 t/s (3 runs each, llama-bench). Build 8452824, CUDA 13.3, driver 595. ttft/ram/power recorded as 0 - not captured by llama-bench; see limitations. | 3403.5 | 172.1 | 0 | 19.0 | 0.0 | 0 |
Quality scores
CODING
0.81/ 0-1
HumanEval pass@1 (carried from bm-002; not re-run)
Limitations
- ◆ttft_ms, ram_used_gb, and power_watts are 0 because llama-bench does not report time-to-first-token, system RAM, or sustained power. VRAM was captured separately via nvidia-smi during a server-mode load. These fields use the site's 0 = 'not measured' sentinel.
- ◆Workload is llama-bench pp512+tg256, NOT the site's workload-pack-v1. Do not cross-compare the 172 tok/s here against workload-pack-v1 generation figures (which use real prompts at varying lengths). This number is for backend/platform speed comparison.
- ◆Dual-GPU tensor split: the 19.0GB VRAM figure is split across two 12GB cards (9.72 + 9.32). This is NOT a single-24GB-card result - the config does not exist on this machine. A single-GPU run would OOM at Q4_K_M + 8K context.
- ◆Mixed GPU SKUs: the two cards are different (RTX 5070 desktop-class + RTX 5070 Ti Laptop). Tensor-split load balancing between unequal-SKU GPUs can introduce minor asymmetry not captured here.
- ◆Model quality (HumanEval 0.81) carried over from bm-002 - same artifact, not re-measured. A fresh HumanEval run is a documented follow-up.
- ◆model_checksum not captured at run time (noted as pending).
- ◆Raw evidence link (/lab/raw/bm-009/) currently resolves to a stub; the /tmp/bm009_bench.log and nvidia-smi captures to be uploaded before or within 30 days of launch.
Corrections
2026-07-18
Initial publication. Supersedes bm-002, whose provisional Victor numbers (52.1 gen / 612.4 prompt tok/s, claimed single-24GB-GPU at CUDA 12.5) were ~3x too slow and based on an incorrect hardware spec. bm-002 is marked superseded; its record is retained for history.