Running locally vs in the cloud
Open-weight models can run on your own hardware at home, trading the cloud's scale and convenience for privacy, control and a fixed cost.
Read first: GPU clouds & neoclouds.
D1 · StoryFor young children and total beginners
You can use an AI over the internet, on someone else's computers. Or you can download some AIs and run them on your own computer at home.
At home, your words stay with you. But your computer is much smaller, so the biggest AIs won't fit.
Like: Cooking at home versus eating at a restaurant.
D2 · PictureFor curious beginners
Cloud (an API or rented GPUs): the largest and newest models, no hardware to buy, pay per token or per hour, but your data leaves your network and the provider sets the rules.
Local (a desktop with one or more consumer GPUs, or a compact AI workstation): only open-weight models, limited mainly by how much fast memory you have, but private, works offline, no per-token bill, and you choose the model, including fine-tuned or less filtered ones.
Quantisation is what makes local practical: storing weights with fewer bits lets a bigger model fit in the same memory, at some cost in quality. A local runtime handles the download and running.
Like: Cooking at home versus eating at a restaurant.
Where it breaks: A home kitchen can cook most dishes a restaurant can. A home machine can't run the largest frontier models at all, and many of those are never released as weights, so local and cloud are often used side by side.
D5 · FrontierFor experts
The simple picture breaks on cost: local looks free per token, but hardware, power and idle time add up, and a busy cloud GPU shared by many users can be cheaper per token.
Privacy also depends on the details: some cloud providers offer strong data controls, and local setups still need patching and care. Which wins depends on usage, data sensitivity and how fast models outgrow home hardware.
Examples as of 2026-10-13
- llama.cpp and Ollama (local runtimes)
- Model APIs and rented GPUs (cloud)