https://localaifrontier.com Reproducible benchmarks, tested home-lab builds, and honest model recommendations — produced from real hardware testing, not rewrites of other people's numbers. en-us Mon, 20 Jul 2026 03:04:43 GMT https://localaifrontier.com/blog/offline-voice-assistant/ https://localaifrontier.com/blog/offline-voice-assistant/ Wed, 29 Jul 2026 00:00:00 GMT The deep-dive on use-case #3. Wire a local speech-to-text model, a local LLM, and a local TTS engine into a voice assistant that works offline, has no wake-word corporation listening, and runs on hardware you own. Architecture, latency budget, and the honest '1–2 seconds' reality. blog https://localaifrontier.com/blog/moe-shift-2026/ https://localaifrontier.com/blog/moe-shift-2026/ Wed, 22 Jul 2026 00:00:00 GMT Every model worth running locally in 2026 — Qwen3-30B-A3B, LFM2.5-8B-A1B, the GLM/Kimi frontier — is MoE. Here's what 'active parameters' actually means, why it's the reason these models fit on your card, and the measured 2.5× speed data behind it. blog https://localaifrontier.com/blog/private-rag-knowledge-base/ https://localaifrontier.com/blog/private-rag-knowledge-base/ Tue, 28 Jul 2026 00:00:00 GMT The deep-dive on use-case #2. Drop contracts, research dumps, or internal docs into a local RAG stack and the model reads, retrieves, and answers — with nothing leaving your disk. Architecture, model choices, the honest limits of local retrieval, and a working starting config. blog https://localaifrontier.com/blog/oss-llm-landscape-july-2026/ https://localaifrontier.com/blog/oss-llm-landscape-july-2026/ Tue, 21 Jul 2026 00:00:00 GMT GLM-5.2, DeepSeek V4, Kimi K2.7, Qwen3.6, MiniMax M3 — every roundup lists them. Almost none tell you which actually run on a card you own. Here's the landscape, sorted by what fits on consumer hardware, with VRAM math and the honest 'API-only' calls. blog https://localaifrontier.com/blog/batch-process-documents-locally/ https://localaifrontier.com/blog/batch-process-documents-locally/ Thu, 30 Jul 2026 00:00:00 GMT The deep-dive on use-case #5. Classify, extract, summarize, or transform a pile of documents headlessly — running the model against a queue, not a chat. Where local AI's cost-at-scale argument actually bites, the throughput math, and the honest 'one stream at a time' limit. blog https://localaifrontier.com/blog/quantization-2026-no-compromise/ https://localaifrontier.com/blog/quantization-2026-no-compromise/ Thu, 23 Jul 2026 00:00:00 GMT In 2024, Q4_K_M meant 'noticeably broken.' In 2026 it's the practical default. We ran the Q4 vs Q6 vs Q8 sweep on Qwen3-Coder-30B — here's the measured quality delta, the VRAM math, and when you actually should step up. blog https://localaifrontier.com/blog/self-host-chatgpt-300/ https://localaifrontier.com/blog/self-host-chatgpt-300/ Sun, 26 Jul 2026 00:00:00 GMT A working private chat assistant — model, web UI, and OpenAI-compatible API — running entirely on hardware you own, for the price of one card. The shortest path from zero to 'it works,' with the honest limits and where to go next. blog https://localaifrontier.com/benchmarks/qwen36-27b-cross-machine-baseline-dual-vs-single-gpu/ https://localaifrontier.com/benchmarks/qwen36-27b-cross-machine-baseline-dual-vs-single-gpu/ Sun, 19 Jul 2026 00:00:00 GMT First cross-machine baseline of the lab. The same model — Qwen3.6-27B at Q4_K_M, identical Ollama digest a50eda8ed977 (byte-identical weights) — run on all four lab machines under controlled conditions (same 37-token prompt, 512 generated tokens, temperature 0, seed 42, one warmup rep). Result: the three CUDA/ROCm machines cluster within ~10% for generation (Ray 26.74, Evo-X2 25.21, Victor 24.28 tok/s), so for a 27B Q4_K_M model dual-12GB NVIDIA is NOT meaningfully faster than single-20GB AMD. The dual-GPU advantage only matters once the model exceeds ~18GB. Jitori's 8.41 tok/s is Vulkan-fallback performance, NOT the Arc B60's real capability (SYCL is broken on Battlemage — see bm-012). This directly answers Research Question RQ-001. benchmark https://localaifrontier.com/blog/cloud-image-generation-apis-compared/ https://localaifrontier.com/blog/cloud-image-generation-apis-compared/ Sun, 19 Jul 2026 00:00:00 GMT Replicate, fal.ai, Midjourney, and OpenAI compared for cloud image generation. The billing models differ more than the models do — per-second, per-image, subscription, and token-bundled each win for different usage patterns. Here's the decision tree. blog https://localaifrontier.com/blog/cloud-video-generation-apis-compared/ https://localaifrontier.com/blog/cloud-video-generation-apis-compared/ Sun, 19 Jul 2026 00:00:00 GMT Cloud video generation has the widest price range in AI media — $0.13 to $3.20 per clip. Runway, fal.ai, Pika, and Sora compared on pricing models, quality, and content control, with a decision tree for which provider wins for which use case. blog https://localaifrontier.com/blog/comfyui-optimization-2026/ https://localaifrontier.com/blog/comfyui-optimization-2026/ Sun, 19 Jul 2026 00:00:00 GMT How to make ComfyUI actually fast on a card you can afford. Dynamic VRAM is the big 2026 leap, but the quantization and attention-backend choices matter as much. Here's what each optimization does, when it helps, and the honest trade-offs. blog https://localaifrontier.com/blog/cross-machine-qwen36-27b-dual-vs-single-gpu/ https://localaifrontier.com/blog/cross-machine-qwen36-27b-dual-vs-single-gpu/ Sun, 19 Jul 2026 00:00:00 GMT We took Qwen3.6-27B at Q4_K_M — byte-identical weights, verified by Ollama digest — and ran it on all four lab machines: dual 12GB NVIDIA, single 20GB AMD, dual 32GB AMD, and a 24GB Intel Arc. The three CUDA/ROCm machines landed within 10% of each other. The honest answer to 'do I need two GPUs?' is: only once the model stops fitting in one. blog https://localaifrontier.com/blog/local-video-generation-wan-hunyuan-cogvideox/ https://localaifrontier.com/blog/local-video-generation-wan-hunyuan-cogvideox/ Sun, 19 Jul 2026 00:00:00 GMT Running Wan, Hunyuan Video, and CogVideoX locally — what's actually runnable on consumer GPUs, what isn't, and the optimizations that help. The honest version, because video is where local hardware hurts most. blog https://localaifrontier.com/blog/local-vs-cloud-ai-media-generation/ https://localaifrontier.com/blog/local-vs-cloud-ai-media-generation/ Sun, 19 Jul 2026 00:00:00 GMT The local-vs-cloud question for AI image and video generation isn't 'which is better.' It's a cost, privacy, and volume calculation — and the answer is different from the one for text. Here's the framework, the break-even math, and where each side actually wins. blog https://localaifrontier.com/blog/run-flux-stable-diffusion-locally/ https://localaifrontier.com/blog/run-flux-stable-diffusion-locally/ Sun, 19 Jul 2026 00:00:00 GMT What it actually takes to run Flux and Stable Diffusion on your own GPU — VRAM floors by model, GPU picks by budget tier, and the five optimizations that make marginal cards viable. No 'best GPU' list — the math. blog https://localaifrontier.com/blog/battlemage-one-year-in/ https://localaifrontier.com/blog/battlemage-one-year-in/ Sat, 25 Jul 2026 00:00:00 GMT A year ago Intel Arc was still a punchline for AI workloads. The Battlemage refresh (B580/B60) is the thing that changed that — and a year of driver, backend, and ecosystem work has made it genuinely good, with caveats. Here's where Arc actually stands for local AI in mid-2026. blog https://localaifrontier.com/benchmarks/qwen3-coder-30b-a3b-victor-cuda-vs-jitori-vulkan/ https://localaifrontier.com/benchmarks/qwen3-coder-30b-a3b-victor-cuda-vs-jitori-vulkan/ Sat, 18 Jul 2026 00:00:00 GMT SUPERSEDED 2026-07-18 — see bm-009. Provisional Victor numbers in this record were ~3x too slow (real: 172 tok/s gen, 3403 tok/s prompt) and based on an incorrect single-24GB-GPU hardware spec. Retained for history only; do not cite. The original summary read: 'CUDA on the NVIDIA reference produces 52.1 tok/s vs. 38.6 tok/s on Arc Vulkan — a 35% generation-speed lead at the same 24GB VRAM tier.' benchmark https://localaifrontier.com/benchmarks/qwen3-coder-30b-a3b-jitori-vs-ray/ https://localaifrontier.com/benchmarks/qwen3-coder-30b-a3b-jitori-vs-ray/ Sat, 18 Jul 2026 00:00:00 GMT Vulkan on Intel Battlemage lands within 12% of dual-Radeon AI PRO R9700 for code generation at Q4_K_M, at roughly half the system cost. Arc is the price-performance leader for sub-$2K builds; multi-GPU AMD wins raw throughput once you accept the complexity. benchmark https://localaifrontier.com/benchmarks/qwen3-6-27b-mtp-long-context-decay-r9700/ https://localaifrontier.com/benchmarks/qwen3-6-27b-mtp-long-context-decay-r9700/ Sat, 18 Jul 2026 00:00:00 GMT MTP speculative decoding on Qwen3.6-27B (single Radeon AI PRO R9700, llama.cpp Vulkan) holds 55-65 tok/s generation across 8K-80K context - degrading roughly 1.3x over that range - while prompt processing falls from ~760 to ~580 tok/s as context fills. The practical finding: MTP keeps working at long context (acceptance 66-75%), but KV-cache pressure steadily erodes both prompt and generation speed. Founder's prior short-prompt runs hit ~75-86 tok/s; the numbers here use real document prompts and are correspondingly a bit lower. benchmark https://localaifrontier.com/benchmarks/lfm25-8b-real-app-evox2-rx7900xt/ https://localaifrontier.com/benchmarks/lfm25-8b-real-app-evox2-rx7900xt/ Sat, 18 Jul 2026 00:00:00 GMT LFM2.5-8B-A1B (Q4_K_M, 8.47B total / ~1.7B active MoE params, 4.79 GiB) measured on TWO GPUs for a clean MoE-vs-GPU comparison: EvoX2's AMD Radeon RX 7900 XT (gfx1100, ROCm) at 269.01 tok/s gen / 7464.85 tok/s prompt using 6.81GB VRAM, and Jitori's Intel Arc Pro B60 (Battlemage, Vulkan) at 68.79 tok/s gen / 1688.53 tok/s prompt. The RX 7900 XT is ~3.9x faster on this MoE than the Arc B60 - the SAME ratio as on the dense Llama-3.1-8B (see bm-008), confirming the GPU gap is consistent across architectures. Companion to bm-011 (the APU iGPU result on the same machine). Closes the small-model and real-app workload gaps. benchmark