Running a large language model on your own PC used to mean wrestling with WSL, Docker, and a pile of dependencies. That's no longer the case. Ollama turns your Windows machine into a private AI workstation with a single installer, no admin rights, and no subscription fees. This guide walks through the entire process, from checking your system to chatting with your first local model.
What Is Ollama?
Ollama is an open-source tool that lets you download and run large language models directly on your computer. Instead of sending your prompts to a cloud API, everything happens locally: your data stays on your machine, you're not paying per token, and once a model is downloaded, it works completely offline. It's a practical option for developers, writers, or anyone who'd rather keep sensitive information off someone else's server.
System Requirements
Before installing, confirm your setup meets the basics:
- Windows 10 (version 1903 or later) or Windows 11, 64-bit
- At least 8GB of RAM
- A GPU is optional but helpful. NVIDIA cards get CUDA acceleration; AMD cards use ROCm or DirectML. Either can speed up responses by 5 to 10 times compared to CPU-only. Without a GPU, Ollama still runs — it just takes longer to respond, especially with larger models.
No administrator account is needed for the standard install, which makes this easier to set up on managed or shared machines.
Step 1: Download the Installer
Go to ollama.com/download/windows and download OllamaSetup.exe. The file is small, around 5MB, with no extra dependencies to install first.
Step 2: Run the Installer
Double-click the downloaded file and click through the setup wizard. There's nothing to configure here — the installer sets everything up in under a minute. Once it finishes, Ollama adds itself to your system tray and starts running as a background service automatically.
Step 3: Verify the Installation
Here's the part that trips people up: open a brand-new PowerShell or Command Prompt window. If you had a terminal open before installing, it won't recognize the updated system PATH, and commands will fail even though the install worked fine.
In the new window, type:
ollama --version
If you see a version number, the install succeeded. If you get a "not recognized" error, close every open terminal and try again in a fresh one. On rare occasions, a full system restart is needed for the PATH change to take effect.
Step 4: Run Your First Model
You have two ways to interact with Ollama.
Command line: Type a command like:
ollama run phi3:mini
This downloads a lightweight, fast model and drops you into an interactive chat session. For something more capable, try ollama run llama3.2 instead. The first run takes a few minutes since it's pulling several gigabytes of model data. After that, it launches instantly and works without an internet connection.
Desktop app: Click the Ollama icon in your system tray to open a built-in chat window. Pick a model from the dropdown — selecting one you haven't downloaded yet triggers the download automatically — and start typing. It behaves like any standard chat interface, but everything runs on your hardware.
Handling the Windows Firewall Prompt
The first time Ollama starts, Windows Firewall may ask for permission. Click Allow access for private networks. This only lets local applications on your machine talk to Ollama at localhost:11434 — it doesn't open anything up to the internet or expose your machine externally.
Quick Troubleshooting
A few common snags and their fixes:
- GPU not detected: Update your NVIDIA or AMD graphics drivers to the latest version.
- Port conflict: Run
netstat -ano | findstr 11434in Command Prompt to find what's using the port, then close it. - Running low on disk space: By default, models are stored in your user folder. Set the
OLLAMA_MODELSenvironment variable to point somewhere else, like a secondary drive, before downloading anything large.
Wrapping Up
The entire process boils down to three moves: download the installer, run it, and pull your first model. A few commands worth keeping handy going forward:
ollama list— see what's installedollama pull— download a new model without running it yetollama rm— remove one to free up space
From here, the main decision is picking the right model for your hardware — smaller models for lighter machines, larger ones if you've got the RAM and GPU to back them up.