How Much VRAM Do You Actually Need for a Local LLM?

The real math behind local LLM VRAM requirements, and how our 64GB and 128GB AI Workstation configs stack up against it.

Every guide to running a local LLM says "it depends on the model." True, but useless on its own. Here’s the actual math, and where our two workstation configs land on it.

The short version

VRAM in GB is roughly: parameters (in billions) × bits per weight ÷ 8. A 7B model at 8-bit needs about 7GB. Drop to 4-bit and it’s about 3.5GB. Add 10-20% on top for context, the conversation history the model is actively holding in memory, and you’ve got a real planning number instead of a guess.

Model size to VRAM, roughly

7B-8B (Llama 3.1 8B, Mistral 7B): about 4-6GB at 4-bit, 8-10GB at 8-bit

13B-14B: about 8-10GB at 4-bit, 14-16GB at 8-bit

20B-35B (gpt-oss-20b, Qwen 32B-class): about 14-22GB at 4-bit, 24-38GB at 8-bit

70B: about 40-45GB at 4-bit, 75-80GB at 8-bit

100B+ MoE-class: 55GB+ at 4-bit, 100GB+ at 8-bit

These numbers are estimates. The real figure shifts with architecture, context length, and quantization method (GGUF Q4_K_M isn’t the same as a plain 4-bit cast). Use this to plan, not as a spec sheet.

Where our two configs land

We build one platform, the AMD Ryzen AI Max+ 395 with unified LPDDR5x memory, in two memory tiers. Here’s what each tier can run:

64GB ($2,699), ~48GB usable: runs the 20B-35B range, think gpt-oss-20b or Qwen 32B-class, at 4-bit or even 8-bit with plenty of room left for context. See the full 64GB build.

128GB ($3,899), ~96GB usable: runs 70B dense models and large MoE models around 100B. MoE models look huge on disk, but only a fraction of their parameters activate per token, so they run well here despite the size. See the full 128GB build.

Neither tier is objectively better. It’s about what fits your model, not which config wins on paper.

Why we don’t chase 8GB consumer GPUs for this

A consumer gaming GPU tops out around 16-24GB of VRAM. That covers the low end fine, but it’s a hard ceiling: no amount of patience gets a 70B model into 16GB. Our unified memory architecture trades some raw GPU speed for a much higher memory ceiling, and for most local-LLM work, memory is the real bottleneck, not speed.

The honest caveat

Quantization isn’t a free lunch. A 4-bit model is smaller and faster than the same model at 8-bit or full precision, but it loses some capability, how much depends on the task and the model family. For drafting, summarizing, RAG-style document search, and coding help, 4-bit is a safe default. For anything where precision actually matters, exact numeric reasoning especially, 8-bit buys back real accuracy at roughly double the memory cost.

Bottom line

Figure out your model class first, what you actually need the thing to do, then size the hardware to match. Not the other way around. If you’re not sure yet, the 64GB vs. 128GB breakdown covers what each config actually does in practice. Once you know your model size, see which tool actually runs it.