Guide

Can you run local AI on an 8 GB Mac?

Yes. Not comfortably, and not all of it. An 8 GB Mac gives you about 4.8 GB of room for model weights before anything else on the machine starts losing. That is enough for a good small model and not enough for a good large one, and the difference between a pleasant experience and an unusable one is mostly about what else you have open.

The short answer

Take a 4-bit model whose file is 3 to 5 GB — in practice a 3B, 4B or 7B. Expect somewhere between 12 and 25 tokens a second, which reads a little slower than a person types. Quit the browser before you start. Do not try to generate images at the same time.

Everything below is why, and what changes if you ignore each part.

The arithmetic

Model weights are memory-mapped and have to stay resident while the model answers. The file size on disk is very close to what they occupy. On top of that sits the KV cache, which grows as the conversation gets longer, plus macOS, plus every application you have not quit. The working rule is 60 percent of installed memory for the weights, with 80 percent as a wall.

These are the same thresholds Slate uses when it labels a model fits, tight or too big before you download it. On an 8 GB machine nothing in its starter catalogue is labelled fits — the smallest entry is 5.6 GB and it is shown as tight, because that is what it is. The full memory guide covers every other RAM size.

What actually fits

Parameter counts are not sizes. What matters is the file, and the file depends on how the model was quantised. At 4-bit, a rough rule is 0.6 GB per billion parameters.

Going below 4-bit to squeeze a bigger model in is the wrong trade. A 4B at Q6 will beat a 14B mangled into 2-bit, because aggressive quantisation damages the weights faster than extra parameters recover — and it damages them exactly where you notice: following instructions, arithmetic, staying coherent past a few paragraphs.

Your browser is also a model

This is the part that makes the difference between "it works" and "my Mac froze". A browser with a dozen tabs open routinely holds 3 to 4 GB. On an 8 GB machine that is the entire comfortable budget, spent before the model has loaded a single byte.

Open Activity Monitor and look at the Memory tab. The graph at the bottom is memory pressure, and it is the only number worth watching: green means the model is in memory, yellow means macOS is compressing to keep it there, red means it is going to disk. A model that generates its first token a minute after you press return is not a slow model. It is a model being read off an SSD.

How fast it will be

Generating a token requires reading every weight, so the ceiling is memory bandwidth divided by model size. Real throughput lands at roughly 60 to 80 percent of that. Base Apple silicon has about 70 GB/s on the M1 and roughly 100 to 120 GB/s from the M2 onward, which makes the estimate easy.

Estimates from the bandwidth arithmetic, not a benchmark of your machine. They hold while memory pressure is green. Once it is not, none of this applies — you are measuring your SSD.

What will not work

Being clear about this is more useful than being encouraging.

Image generation alongside chat. An image model is a second set of weights, and a small one is still a couple of gigabytes. On 8 GB you run one or the other, not both, and even alone it will lean on CPU offload and take its time.

Long context. The KV cache is not free and it grows with the conversation. A 32k-token context can cost a gigabyte or more on its own, which on this machine is a fifth of everything you have. Long documents are where an 8 GB Mac stops.

Several models at once. Slate's Roundtable answers a question with two or three models side by side. Two 4B models will fit; anything larger will not. This is a memory limit, not a licensing one.

What works genuinely well

Speech is the pleasant surprise. Transcription and text-to-speech models are a few hundred megabytes, not gigabytes, so dictation and read-aloud run fine on 8 GB and leave the rest of the budget alone.

So does most short-form text work: rewriting an email, tightening a paragraph, summarising a page, pulling structured fields out of a mess, explaining a function, naming things. A 4B model does that competently and instantly, and it does it without an account, a network connection or a bill.

If you are buying

Apple has stopped selling 8 GB Macs, so this page is about the M1 and M2 machines already on desks rather than about a decision you are making. If you are making one: 16 GB is the floor, 24 GB is where the useful models stop being a compromise, and memory is soldered, so it is the one thing you cannot fix later.

But do not replace a working Mac over this. Run a 4B model for a week first. If it does your work, you have saved a lot of money; if it does not, you now know exactly what you are buying more memory for, which is a much better position than guessing.