How to install Ollama on a VPS

Run your own open-source LLM on a server you control. This walks through it end to end, from a fresh VPS to a working API, in about 15 minutes.

Updated 22 July 2026 · About a 15 minute setup

Ollama is the simplest way to run open-source language models yourself. It downloads a model, runs it, and exposes an OpenAI-compatible API, all from one command. Best of all, small and mid-size models run on an ordinary CPU VPS, so you do not need a GPU to get started.

By the end of this guide you will have a model like Llama 3.1 or Mistral answering requests on your own server. For a full picture of which models fit which plan, see what open-source AI runs on a VPS.

What you need

A Linux VPS. We recommend a KVM VPS with at least 8 GB of RAM for a 7B or 8B model. KVM is best because you get full control and can add Docker later.
Ubuntu 22.04 or 24.04 (these steps work on most Debian-based systems).
Root or sudo access over SSH.

Step 1: Provision a VPS

Start with a KVM VPS sized for the model you want. An 8 GB plan comfortably runs a 7B or 8B model. If you only need a small model, 4 GB is enough; for bigger models or several at once, size up. See the what runs on each VPS size table to match a model to a plan, and pick one on the VPS plans page.

Step 2: Connect over SSH and update

ssh root@YOUR_SERVER_IP
apt update && apt upgrade -y

Step 3: Install Ollama

Ollama provides an official install script. One command installs it and sets it up as a background service.

curl -fsSL https://ollama.com/install.sh | sh

When it finishes, confirm it is installed:

ollama --version

Step 4: Download and run a model

Pull and run a model in one step. The first run downloads the model, which can take a few minutes depending on its size.

ollama run llama3.1

You will drop into an interactive prompt. Type a question, and type /bye to exit. Prefer a different model? Try ollama run mistral or a smaller, faster option like ollama run llama3.2:3b.

Step 5: Use the API

Ollama runs a local API on port 11434. Test it with a simple request:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt": "Explain what a VPS is in one sentence.",
  "stream": false
}'

You will get a JSON response with the model's answer. Because the API is OpenAI-compatible, most existing tools and libraries can point at your server with only a base-URL change.

Security first. By default Ollama listens on localhost only, which is what you want. Do not expose port 11434 to the public internet without protection. If you need remote access, put it behind a firewall and a reverse proxy such as Nginx or Caddy with authentication and HTTPS.

Step 6: Add a private ChatGPT UI (optional)

For a friendly web interface, run Open WebUI with Docker. On a KVM VPS with Docker installed, this one command starts it and connects it to the Ollama you just set up:

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui --restart always \
  ghcr.io/open-webui/open-webui:main

Then open http://YOUR_SERVER_IP:3000, create the first account, and you have your own private chat interface. Put it behind HTTPS before real use.

What to run next

With Ollama in place you can build on it: connect it to n8n for AI automation, add a vector database like Qdrant for retrieval-augmented generation, or run several models at once on a larger plan. If responses feel slow, that is CPU inference at work. Expect roughly 4 to 15 tokens per second for a 7B or 8B model, which suits personal tools and low-traffic APIs. For heavier needs, size up your VPS.

Common questions

Ollama on a VPS, answered.

Do I need a GPU to run Ollama?

No, not for small and mid-size models. Models in the 1B to 8B range run on a CPU VPS. A GPU only becomes necessary for large models at high speed or for image generation.

How much RAM do I need?

About 8 GB of RAM runs a 7B or 8B model like Mistral 7B or Llama 3.1 8B comfortably. Smaller 1B to 3B models fit in 2 to 4 GB.

Which model should I start with?

Mistral 7B or Llama 3.1 8B are good general-purpose starting points on an 8 GB VPS. Phi-3 Mini or Llama 3.2 3B are faster and fit smaller plans.

Is CPU inference fast enough?

Expect roughly 4 to 15 tokens per second for a 7B or 8B model on a CPU VPS. That is fine for personal use, internal tools, automation and low-traffic APIs, but not high-traffic real-time serving.

Ready to run your own AI?

See what runs on each VPS size, then start on a KVM plan built for it.

Run AI on a VPS See VPS plans