Local LLMs

Running large language models on your own computer: links to Ollama and a tool for checking which models your hardware can handle, plus step-by-step Ollama and Mistral setup for macOS and Windows.

Local large language models run on your own computer, so documents and data you give them stay on your machine. Use the links below to find the tools, or follow the setup steps for Ollama further down this page.

Links

• Ollama — Downloads and runs open language models on your own computer.
• Mistral model — The Mistral model’s page in the Ollama library.
• Ollama on GitHub — Source code and documentation for Ollama, which downloads and runs open models locally.
• llmfit — Command-line tool that checks which models will run on your hardware.

Set up Ollama

Follow the steps below to install Ollama, a local model runner, and download the Mistral model. This setup allows you to run a large language model locally on your computer without an API key.

macOS installation

Option A: Standard install

1. Go to the Ollama download page
2. Click Download for macOS
3. Open the downloaded .dmg file
4. Drag Ollama into your Applications folder
5. Open Ollama once to start the background service

Option B: Terminal install

Open Terminal and run:

curl -fsSL https://ollama.com/install.sh | sh

Verify installation

Run:

ollama --version

Windows installation

Option A: Standard install

1. Go to the Ollama download page
2. Click Download for Windows
3. Run the installer (OllamaSetup.exe)
4. Follow the installation steps

Option B: PowerShell install

Open PowerShell and run:

irm https://ollama.com/install.ps1 | iex

Verify installation

Open Command Prompt, PowerShell, or Windows Terminal and run:

ollama --version

Download and run Mistral

Run:

ollama run mistral

The first time you run this command, Ollama will download the Mistral model (several GB). After the download finishes, an interactive chat session will start.

Test prompt

After starting Mistral, type directly into the terminal:

Explain linear regression in three bullet points.

You should receive a generated response. To exit, press Ctrl + D or type /bye.

Hardware requirements

Minimum:

• 8 GB RAM

Recommended:

• 16 GB RAM

Notes:

• No GPU required
• Several GB of free disk space needed

Useful commands

List installed models:

ollama list

Download without running:

ollama pull mistral

Remove a model:

ollama rm mistral

Troubleshooting

“ollama not found”

• Restart Terminal or your computer

Slow performance

• Close other applications
• Ensure enough available RAM

Download stuck

• Press Ctrl + C
• Run again: ollama run mistral

Model will not start

• Restart Ollama or reboot your system