Will it fit? VRAM math for local LLMs

The arithmetic behind Model Fit, with worked examples. Once you can do this in your head, you stop needing the tool.

The formula

Memory footprint, simplified:

footprint = weights + kv_cache + runtime_overhead

weights_gb       = params_in_billions × bytes_per_param(quant)
kv_cache_gb      = (context_tokens / 1000) × 0.18   # conservative
runtime_overhead = ~1.0 GB                            # llama.cpp baseline

Worked example: Qwen3-Coder-30B-A3B at Q4_K_M, 8K context

weights       = 30 × 0.55         = 16.5 GB
kv_cache      = 8.192 × 0.18      = 1.47 GB
runtime       =                   1.0 GB
-----------------------------------------
total                            ≈ 19.0 GB

On a 24GB card (Arc B60 Pro, RX 7900 XTX, RTX 4090), this fits with ~5GB headroom — "comfortably" in our estimator. Our measured result was 18.1 GB VRAM used, close to the estimate.

Worked example: 70B at Q4_K_M, 8K context

weights       = 70 × 0.55         = 38.5 GB
kv_cache      = 8.192 × 0.18      = 1.47 GB
runtime       =                   1.0 GB
-----------------------------------------
total                            ≈ 41.0 GB

Doesn't fit any single consumer 24GB card. Options: (a) multi-GPU split across two 24GB cards, (b) drop to Q3_K_M (~31.5GB — still over 24GB, requires offload), (c) accept RAM offload with major speed penalty.

Ready to plug in your numbers? Open Model Fit →