Last time I looked, my conclusion was that it would not be possible to run a local LLM on my MacBook Air M2/24 GB RAM, but in another chat room, I found this:
“I use 2GB up to 12GB models on my 16GB macbook air M2 with little problem. It is amazing to have a condensed version of most human knowledge running on an ultralite laptop from several years ago. You have more memory than me so this will be even smoother. I am more used to LM Studio, but newer engines like oMLX are even faster / more memory efficient. I use Gemma 4 models, but suspect the newest Qwen’s are even better as they support cool new techniques developed by Deepseek to speed up local models etc.”
Before I dig into this, I would love to hear your opinion. How is this really possible? What engine and version of LLM should I choose so it runs smoothly?
I am totally inexperienced in local LLMs, so I am not sure what I should do.
Your opinion is much appreciated!
I’m curious about this also as I have the same laptop and would be interested in seeing how a local model runs. What do you use for the front end if you do run a local model?
Probably the easiest thing to do is install ollama. By default you’ll have a desktop chat-like application that you can use to interact with local models that you can download. Which models would work best for you depends heavily on your system’s processor, memory bandwidth, and unified memory.
Edit to add: For 24GB unified memory, you probably want to try models that don’t exceed 20 billion parameters with 4 bit quantization. Those will be pushing the upper limits of what your computer’s memory can manage.
Also, mixture of expert (MoE) models will be much faster than dense models.
(It doesn’t matter too much if you don’t know what those things mean right away: The model descriptions in the various libraries will tell you which models fit those limits. You’ll figure out the meanings of the jargon if you get into it and start to do a bit of research 
)