You've picked a model, installed Ollama or LM Studio, and hit run. Either it loads and crawls along at a couple of words a second, or it refuses to load at all. Somewhere between the spec sheet and your desktop, something went wrong — and it almost always comes down to one number you didn't pay enough attention to: VRAM.

Local LLMs don't care about your CPU, your RAM, or how many cores your processor has. They care about how much video memory your GPU has, full stop. Get that number right and a local model feels almost as fast as a cloud API. Get it wrong and you'll wonder why anyone bothers running AI at home.

Why VRAM Is the Only Spec That Actually Matters

Here's the part most buying guides skip: generating text with an LLM is a memory-bandwidth problem, not a raw-power problem. After the first word, the GPU spends most of its time waiting on memory reads, not doing math. That's why a card with dramatically higher compute numbers but the same memory speed generates tokens at nearly the same pace as a "weaker" card — the horsepower isn't the bottleneck.

The real cliff shows up when a model doesn't fit in VRAM at all. Once it spills into system RAM, performance doesn't dip a little — it falls off a cliff. A flagship card running a large model entirely in VRAM can hit 45+ tokens per second. The exact same setup, offloading even part of that model to system RAM, can drop to one or two tokens per second — slower than you can read. There's no in-between; it's fast or it's frustrating.

What "Minimum" Actually Means This Year

A couple of years ago, 8GB of VRAM was a reasonable starting point for local AI. That's no longer true. Models have gotten smarter and heavier at the same time, and 16GB is a more honest floor if you want to run something worth using, not just something that technically boots.

The rough math for the standard quantization level (Q4_K_M) most people run looks like this: an 8-billion-parameter model needs around 6GB, a 34-billion-parameter model needs about 20GB, and a 70-billion-parameter model needs roughly 42GB. Whatever number you see for a model's file size on Ollama or Hugging Face, treat that as your VRAM floor, then add another 10-20% on top for overhead. That buffer is what keeps a model from crashing the moment you open a longer conversation.

Matching the GPU to the Model You Actually Want

Skip the marketing tiers and match your GPU to the model size you care about:

  • 7-8B models (Mistral, Phi-4 Mini): around 8GB
  • 13-14B models (Qwen3 14B): 12-16GB
  • 24-32B models (Gemma, DeepSeek-R1 32B): 16-24GB
  • 70B-class models (Llama 3.3): 40GB+, usually split across two GPUs

Six Real GPUs, From Entry-Level to Flagship

  • Intel Arc B580 (12GB) — the cheapest legitimate way in. It'll run 7B models fine, but you'll hit its ceiling fast if you try to go bigger.
  • RTX 4060 Ti 16GB — this is the card worth calling the real minimum. Not the lowest number that technically works, but the one that won't leave you frustrated a week later.
  • RTX 5070 Ti 16GB — the modern take on the same tier, with newer memory that pushes tokens out faster on 7B-14B models.
  • RTX 3090 (24GB, used market) — still the best value in local AI. It's an older card, but used units are the cheapest route into the 24-32B class.
  • RTX 4090 (24GB) — same VRAM ceiling as the 3090, meaningfully faster, for anyone who doesn't want to wait out current-gen pricing.
  • RTX 5090 (32GB) — the flagship, and the first consumer card that can hold a 45B+ model entirely on-card without any tricks.

What About Apple Silicon?

Worth a mention if you're not tied to a GPU: Apple's unified memory works like VRAM for local models. A Mac with enough shared memory — say, 128GB on a high-end M-series chip — can run a 70B model at usable speeds without any GPU at all. It's a genuinely different path to the same result.

So, What's the Real Minimum?

Technically, 8GB gets a small model running. Practically, that's not the number to build around. If you want a setup that runs the models people actually recommend — without constant compromises — 16GB is the real starting line, and 24GB is where things start feeling comfortable rather than tight.

Before you buy anything, work backward: pick the model size you actually want to run, match it to the VRAM tiers above, and let that number — not the marketing name on the box — decide the card.