Most people meet AI through a website: you type into ChatGPT, Gemini or Claude, and your words travel to a data center and back. That works well, but it has trade-offs. You need an internet connection, heavy use often means a monthly subscription, and anything you paste leaves your machine.
There is another option. Open AI models, the kind whose files anyone can download, have become good enough for everyday tasks, and a free tool called Ollama makes running them on your own computer close to as easy as installing any other app. This guide walks through what Ollama is, whether your computer can handle it, how to install it on Windows, macOS or Linux, and how to pick your first model.
What Ollama actually does
Ollama is a small program that does three jobs. It downloads open models from its library, runs them on your hardware (using your graphics card if you have a suitable one, or your processor and memory if you do not), and gives you two ways to talk to them: a chat window in the terminal and a local API that other apps can connect to.
That last part matters more than it sounds. Because the API runs at http://localhost:11434 on your own machine, note-taking apps, code editors and browser extensions that support Ollama can use your local model instead of a paid cloud service. According to Ollama's documentation, local use needs no API key at all.
What Ollama is not: it does not make a laptop as powerful as a data center. The largest frontier models still run only in the cloud. Local models are best for drafting, summarizing, rewriting, brainstorming, simple coding help and working with private documents, not for the hardest reasoning problems.
Can your computer run it?
The honest answer depends on which model you want. A model's size is described in parameters (for example 4B means about four billion), and bigger models need more memory. A rough guide that holds across most sources:
- 8 GB of RAM: small models in the 1B to 4B range. Fine for short answers, rewriting and summaries.
- 16 GB of RAM: mid-size models around 7B to 8B run comfortably on a modern laptop. This is the sweet spot for most people.
- 32 GB or more, plus a strong GPU: needed for 13B to 30B-class models at usable speeds.
Graphics memory (VRAM) is the biggest factor for speed. Ollama's quickstart, which uses Google's Gemma 4 E2B model as its example, recommends about 8 GB of available VRAM for that model, or the equivalent unified memory on an Apple Silicon Mac. If you have less, Ollama can fall back to normal system RAM, but responses will be slower. Longer conversations and bigger documents (a larger "context window") also need more memory.
Disk space is the other thing people forget. On Windows the program itself needs roughly 4 GB, and each model is a separate download: the Gemma 4 E2B example is about 7.2 GB, and larger models run to tens of gigabytes.
Windows requirements in detail
Ollama's Windows documentation lists Windows 10 22H2 or newer (Home or Pro). For NVIDIA graphics cards it asks for driver version 551.61 or newer. AMD Radeon cards can use Vulkan acceleration, which is on by default, or ROCm with a compatible driver. No graphics card at all? It still works on the processor alone, just more slowly.
Installing Ollama
On Windows and macOS
- Go to the official download page at ollama.com and choose your operating system. Download only from the official site; copies hosted elsewhere are a common way to spread malware.
- Run the installer. On Windows it installs into your user folder and does not need administrator rights.
- Open the Ollama app. It runs quietly in the background (on Windows you will see it in the system tray).
On Linux
Linux users install it with the one-line script published on ollama.com. If the background service is not running after installation, start it with ollama serve.
Low on space? Move the model folder first
If your main drive is small, tell Ollama to store models elsewhere before you download anything. On Windows, open the settings for environment variables for your account, create a variable named OLLAMA_MODELS that points to a folder on a larger drive, then quit and restart Ollama so the change takes effect.
Your first conversation
Open a terminal (PowerShell on Windows, Terminal on macOS) and download a model. Ollama's own quickstart uses this example:
ollama pull gemma4:e2bWhen the download finishes, start chatting:
ollama run gemma4:e2bYou will get a prompt where you can type questions. Type /bye to leave. Two other commands are worth knowing on day one: ollama list shows the models you have downloaded, and ollama ps shows which ones are currently loaded in memory.
If your machine struggles, with long pauses or a fan that never stops, switch to a smaller model from the Ollama library. If it feels fast, try a larger one. There is no penalty for experimenting; you can delete a model you do not need with ollama rm followed by its name.
Choosing a model
The Ollama library lists hundreds of models, which is overwhelming at first. A simple way to choose:
- Start small. Pick a model in the 2B to 4B range from a well-known publisher such as Google (Gemma), Meta (Llama), Alibaba (Qwen) or Mistral. These download quickly and show you what your hardware can do.
- Match the task. Some models are tuned for coding, some for following instructions in chat, some can read images. The model page on ollama.com says what each is for.
- Check the size tag. The same model often comes in several sizes. The download size listed on the model page is a good proxy for how much memory it will need.
- Read the license. "Open" does not always mean "do anything you like". If you plan to use a model in a business, check its license terms.
What local models are good at, and where they fall short
Running a model locally changes what you can comfortably use AI for. Because nothing leaves your machine, it is a reasonable choice for drafting replies to private emails, summarizing internal notes, or asking questions about documents you would not paste into a website. It also works on a plane or anywhere without a connection, and it costs nothing per question.
The limits are just as real. Smaller models make more factual mistakes and can state wrong information confidently, so treat their answers as a first draft, not a source. Most local models also cannot browse the web, so they know nothing that happened after their training data was collected. And on modest hardware, long documents can be slow to process.
A practical pattern many people settle into: use a local model for private, routine work, and a cloud assistant for research, up-to-date questions and the hardest problems.
Connecting other apps
Once Ollama is running, anything that can talk to its local API can use your model. Ollama's documentation lists endpoints compatible with common formats, including /v1/chat/completions, which many tools built for OpenAI's API already understand. In practice that means you can often point a writing tool, a code editor extension or a self-hosted chat interface at http://localhost:11434 and keep everything on your own machine. Check each app's settings for an "Ollama" or "local model" option.
Is it free?
Running models on your own hardware with Ollama is free. Ollama also sells an optional paid plan that runs larger models on its cloud servers, which is useful if a model does not fit in your memory, but at that point your data is no longer staying local. For the privacy and offline benefits this guide is about, the free local setup is all you need.
Quick checklist
- Check your RAM and graphics memory, then choose a model size to match.
- Make sure you have enough disk space, or set
OLLAMA_MODELSto a bigger drive first. - Install only from ollama.com.
- Run
ollama pullandollama runwith a small model, then scale up. - Double-check important facts; local models make mistakes too.
Comments (0)
To comment, please login or register.
No comments yet. Be the first to comment!