Skip to main content
Skip to main content
The ThreadCat mark: a cat head holding an orange and teal swirl, two threads in one orbit.ThreadCat
Repository

Local model

Run a model on your own computer

A guide for people who have never done this. No coding or terminal experience needed. A local model runs on your computer: nothing you type leaves the machine, there is no account, and there is no subscription. The "brain" of the AI is a file on your disk, not a service someone else can change or shut off.

Why a local model

  • Privacy. Nothing you type leaves your computer. No record kept by anyone but you.
  • Cost. The model files are free. After the download, chatting costs only the electricity your laptop already uses.
  • Your files stay yours. Your history and notes live in plain files you own. This is how our own research tools work.

The steps

  1. Download Ollama from ollama.com. It is free, and it handles everything else. Install it like any app.
  2. Open a terminal. On a Mac: Applications, then Utilities, then Terminal. On Windows: click Start, type "terminal", press enter.
  3. Type one line and press enter:
    ollama run gemma3:4b
    That downloads the model (a few GB; get a coffee) and starts a conversation. The next time, there is no download; it just opens.
  4. Talk to it. Type a message, press enter. When you are done, type /bye.
  5. Optional, the ThreadCat part: run our observer harness against that model and watch what it claims about itself over time. Our tools page explains the project.

Other models that fit a small laptop: llama3.2:3b, qwen3:4b, phi3:mini. Swap the name in the command. All free.

What a local model cannot do

Be honest about this, the same standard we apply everywhere: a 4 GB model is noticeably less capable than the big cloud assistants. It forgets things, it reasons less sharply, and it will still occasionally invent sources (our fabrication checker exists partly because of that). It is the right tool for privacy and for everyday chat; it is not the right tool for heavy research work.

The environmental receipts

Cloud AI has real energy and water costs, and inference (the part you use) is a growing share of them. The studies below are about data centers: buildings full of computers that must be powered and cooled around the clock, wherever you are and whatever time you click.

  • Water. Training the GPT-3 model in Microsoft's U.S. data centers directly evaporated an estimated 700,000 liters of clean freshwater; global AI demand is projected to drive 4.2 to 6.6 billion cubic meters of water withdrawal in 2027, more than the annual water withdrawal of 4 to 6 Denmarks. Source: Li, Yang, Islam, and Ren, Making AI Less "Thirsty", Communications of the ACM (arXiv 2304.03271).
  • Energy and carbon. Multi-purpose generative models are orders of magnitude more expensive in energy and emissions per 1,000 inferences than small single-task models, even when the models have the same parameter count. Source: Luccioni, Jernite, and Strubell, Power Hungry Processing, ACM FAccT 2024 (arXiv 2311.16863).
  • The pattern was named in 2019. Strubell, Ganesh, and McCallum quantified the energy and carbon cost of training large language models and made the equity argument that changed the field's behavior (arXiv 1906.02243, ACL 2019).
  • Where this goes. MIT researchers expect the electricity demands of inference, not training, to dominate AI's energy use as generative models become ubiquitous (MIT News, January 2025).

What this means for a laptop, stated carefully: a small local model answering your questions draws roughly laptop-charger levels of power, produces no cooling-water demand, and adds nothing to a data center's footprint. We are notclaiming a laptop beats a hyperscale data center per token. The honest claims are smaller and sturdier: small models are dramatically cheaper to run than frontier-scale ones, the per-conversation water cost is zero, the energy is visible and on your own meter, and the hardware is already built, already plugged in, and already yours.

Sources

  • Li, Yang, Islam, Ren (2023, rev. 2025), Making AI Less "Thirsty", Communications of the ACM. arXiv 2304.03271
  • Luccioni, Jernite, Strubell (2024), Power Hungry Processing, ACM FAccT 2024. arXiv 2311.16863
  • Strubell, Ganesh, McCallum (2019), Energy and Policy Considerations for Deep Learning in NLP, ACL 2019. arXiv 1906.02243
  • MIT News (2025), Explained: Generative AI's environmental impact. news.mit.edu
  • Ollama, the installer used in this guide. ollama.com

Check us the way we ask you to check everything: click the links. If a number on this page does not match its receipt, tell us; that is the project working.

Written by the ThreadCat project. Every claim links to a receipt; check us.

Full guide in the project repository.

Curious what all this cost to build? Every dollar is itemized in our public cost ledger, including the costs of building this site: cost ledger.