5 things you can do with a local LLM today

Most "what can AI do for you" lists read like a venture pitch. This one is narrower: five workflows you can actually stand up on a local model this week, what each is good at, and the honest limit. All of them run on the $300 setup from our last post.

1. A private coding agent

This is the killer app for local AI in 2026. Run a 30B coder model (Qwen3-Coder-30B-A3B is the current default) through a tool like Continue, Aider, or your editor's local-model integration. It writes code, refactors across files, explains unfamiliar codebases, and writes your tests.

• Good at: routine boilerplate, refactors with clear specs, explaining code in plain language, generating test cases.
• Falls down on: very large repos (context window limits), novel algorithm design, anything requiring the latest library APIs the model was trained before.
• Why local matters: your source code stays on your machine. No accidental leaks through an IDE plugin phoning home.

2. Document Q&A on sensitive files

Drop a 200-page contract, a stack of medical records, or a confidential research dump into a local RAG setup. The model reads, summarizes, and answers questions without anything leaving your disk.

• Good at: "what does this contract say about termination clauses," summarizing long PDFs, finding contradictions between documents.
• Falls down on: precise numerical reasoning across many documents, anything requiring citations the embedding step can't surface cleanly.
• Why local matters: the alternative is uploading sensitive documents to a third-party endpoint. For anything under NDA, HIPAA, or attorney-client privilege, that's not on the table.

3. Voice assistant without the cloud

Wire a local model to Whisper (also local) for speech-to-text and a local TTS engine for output. You now have a voice assistant that works offline, has no wake-word corporation listening in, and can be pointed at any task you can prompt.

• Good at: hands-free notes, dictation, controlling local tools and scripts, kitchen-counter questions.
• Falls down on: the latency budget is tight — local STT → LLM → TTS can hit 1–2 seconds end to end, which feels less snappy than the commercial assistants.
• Why local matters: your voice recordings stay in the room they were spoken in.

4. Private chat and brainstorming

Use the model directly as a chat partner — for drafting, for thinking through problems, for roleplaying scenarios, for therapy-adjacent conversation — without a logged, retained, possibly-reviewed transcript sitting on someone else's server.

• Good at: drafting and rewriting, talking through decisions, low-stakes creative work, repetitive productivity tasks (summarize this email, draft this reply).
• Falls down on: the very frontier of reasoning — a local 30B model is not GPT-5-class on hard logic puzzles.
• Why local matters: the single most underrated property of a local model is that no one is keeping your transcripts. For personal, sensitive, or just embarrassing conversations, that's the whole feature.

5. Automation and batch processing

Run the model headless against a queue: classify and route thousands of support tickets, extract structured data from a pile of PDFs, generate alt text for an image library, summarize a year of meeting transcripts. Anything you'd happily pay an API to do at volume, you can do locally for the cost of electricity.

• Good at: high-volume, repetitive transformation tasks where quality bar is "good enough" rather than "perfect."
• Falls down on: throughput. A single local GPU processes one request at a time at usable speed. If you need to finish a million-item batch by tomorrow morning, you need a server farm, not a workstation.
• Why local matters: the break-even math. At high volume, cloud API costs scale linearly forever; a local GPU's cost is fixed the day you buy it.

The pattern across all five

None of these require a frontier model. All five run on a single 12GB GPU with a 30B-class model. The constraint that used to gate "what can local AI do" — model quality — is no longer the binding one for most real work. The binding constraints now are VRAM and your willingness to do the integration work.

That's the shift that makes 2026 different. The model is good enough. The hardware is cheap enough. The remaining question is just which workflow you point it at first.

If any of these landed for you, Start Here walks through getting a model running in under an hour. The hardware guide covers what to buy if you don't already have a GPU.