You can run a genuinely useful AI assistant on a VPS you control, with no monthly per-seat bill and no customer data leaving your box. The stack is Ollama (runs the language model locally) plus OpenClaw (a chat interface and agent layer on top). Give it a few hours, a VPS with enough RAM, and you have a private assistant your whole team can use.
The honest catch, which most tutorials skip: quality depends on model size, and model size depends on memory. A tiny model on a cheap box will disappoint you. Get the sizing right and it's a different story.
This walkthrough covers what to install, how much RAM you actually need, and where self-hosting wins over a SaaS subscription for an EU business that cares about where data lives.
Why self-host AI instead of paying per seat?
Self-hosting makes sense when you handle customer data you'd rather not send to a third-party model provider, and when a flat VPS cost beats stacking up per-user SaaS fees. Those two reasons carry the whole decision.
On the privacy side: with Ollama, prompts and responses stay on your server. Nothing gets logged by an external vendor, nothing gets used to train someone else's model, and you're not depending on a US provider's data-processing terms to satisfy GDPR. If your VPS sits in the EU, your customer data sits in the EU. That's a clean story to tell an auditor or a nervous client.
On the cost side: a team AI seat from a big provider runs €20–30 per person per month. Ten people and you're at €2,400–3,600 a year, every year. A VPS capable of running a solid local model is a flat monthly figure regardless of how many people use it. The break-even comes fast once you're past a handful of seats.
Self-hosting is not the move if you need the absolute strongest reasoning available and you're comfortable with a SaaS provider's terms. Local models are good, not magic. Be honest with yourself about the job before you commit.
How much RAM and GPU do you really need?
As a rule of thumb, a model needs roughly its parameter count in gigabytes of RAM at 4-bit quantisation, plus headroom for the OS and context. A 7B model wants about 8 GB free; a 13B wants around 16 GB; a 70B is a different league entirely.
Here's the reality without the hype. You do not need a GPU to start. Ollama runs on CPU perfectly well for smaller models, it's just slower, generating a handful of tokens per second instead of dozens. For an internal assistant that drafts emails and answers questions, CPU-only is genuinely fine.
| Model size | Min RAM (4-bit) | Runs on CPU? | Good for |
|---|---|---|---|
| 3B (e.g. Llama 3.2 3B) | 4–6 GB | Yes, snappy | Simple Q&A, drafting |
| 7–8B (Llama 3.1 8B, Mistral) | 8–10 GB | Yes, usable | The sweet spot for SMBs |
| 13–14B | 16 GB | Yes, slow on CPU | Better reasoning |
| 70B | 48 GB+ | Needs a GPU realistically | Heavy work only |
For most small businesses, an 8B model on a VPS with 16 GB RAM hits the balance: room for the model, the OS, OpenClaw, and a couple of concurrent chats. Start there before you spend money on GPU horsepower you might never need. If you're on TPC Hosting, our engineers can tell you whether your current plan has the headroom or whether a bump makes sense before you install anything.
Installing Ollama and pulling your first model
The install is one command, and you'll have a model answering questions within ten minutes. Ollama handles the download, quantisation and serving for you.
On a fresh Ubuntu 22.04 or 24.04 VPS, SSH in and run:
- Install Ollama:
curl -fsSL https://ollama.com/install.sh | sh - Pull a model:
ollama pull llama3.1:8b - Test it:
ollama run llama3.1:8b "Summarise this in one line: ..."
Ollama runs as a systemd service listening on 127.0.0.1:11434. Leave it bound to localhost. Do not expose port 11434 to the internet directly. It has no authentication, so an open port is an open door to your server's compute. OpenClaw will talk to it over that local address, and anything external should go through a reverse proxy with auth in front.
If you want the model to survive reboots and stay warm, set OLLAMA_KEEP_ALIVE=-1 in the service environment so it doesn't unload after each request. That trades a bit of idle RAM for much faster first responses.
Adding OpenClaw as your team's interface
OpenClaw gives your team a browser-based chat and agent layer that points at your local Ollama instance, so nobody needs SSH access to use the assistant. It's the difference between a developer toy and something your sales team will actually open.
Run it with Docker, which keeps it isolated and easy to update:
- Install Docker:
curl -fsSL https://get.docker.com | sh - Start the container, pointing it at Ollama on the host with an environment variable for the base URL (
http://host.docker.internal:11434or your host's Docker bridge IP). - Map its web port to localhost only, then reach it through your reverse proxy.
Once it's up, create user accounts for your team, pick a default model, and you're running. You can point OpenClaw at several models and let people switch, one for quick drafting, a larger one for anything that needs more careful reasoning.
Locking it down before anyone else touches it
Put OpenClaw behind HTTPS with real authentication before you share the URL, and never let Ollama's API face the open internet. That's the whole security story, and it's short because the risks are concentrated in two places.
A practical checklist:
- Reverse proxy with TLS: Nginx or Caddy in front of OpenClaw. Caddy gets you an automatic Let's Encrypt certificate with about three lines of config.
- Firewall:
ufw allow 22,ufw allow 80,ufw allow 443, thenufw enable. Nothing else needs to be open. - Keep 11434 internal: confirm with
ss -tlnp | grep 11434that it's bound to 127.0.0.1. - Strong auth on OpenClaw: real passwords, and disable open sign-up so strangers can't self-register.
- Back up your config and chat data so a rebuild is painless.
Because this box now handles customer conversations, treat it like any system that touches personal data: patch it, restrict SSH to keys, and know where it physically sits. A TPC VPS is EU-hosted and GDPR-friendly, which keeps that last point simple, and our support engineers are real people available 24/7 if a config fights back. If the whole thing turns out not to fit your workflow, you've got 30 days to back out.
Is a local model good enough for real work?
For summarising, drafting, answering questions from your own documents, and routine internal tasks, an 8B–13B local model is genuinely good enough. For the hardest reasoning and long complex chains, hosted frontier models still have an edge, and pretending otherwise would waste your time.
The winning pattern for most SMBs is to run the local model as the default for the 90% of everyday tasks that don't need frontier capability, keeping that traffic private and free of per-seat cost. Reach for a paid API only for the rare heavy job. You get the privacy and the flat bill where it matters, and you're not paying premium rates for work a local model handles fine.
Try it before you judge it. Pull an 8B model, feed it a real task from your week, and see how it does. Most people are surprised how far a well-sized local model goes.
FAQ
What's the minimum VPS spec to run OpenClaw and Ollama usefully?
A VPS with 16 GB RAM and a modern multi-core CPU comfortably runs an 8B model plus OpenClaw. You can start smaller with a 3B model on 4–6 GB, but 16 GB gives you the sweet-spot model quality without needing a GPU.
Do I need a GPU for local AI?
No, a GPU is optional for smaller models. Ollama runs 3B to 13B models on CPU at usable speeds for internal tasks; a GPU only becomes necessary when you want large models or fast responses under heavy concurrent use.
Is a self-hosted AI setup GDPR-compliant?
Self-hosting on an EU VPS keeps prompts and customer data on your own server, which removes the third-party data transfer that complicates GDPR. You still need to secure the box and document your processing, but you avoid sending personal data to an external model provider.
How does the cost compare to a SaaS AI subscription?
A single VPS serves your whole team for a flat monthly fee, while SaaS assistants charge €20–30 per user per month. Past roughly five seats, self-hosting is usually cheaper, and the gap widens as your team grows.
Can I use both a local model and a paid API together?
Yes, and it's the smart setup for most SMBs. Run the local model as the default for everyday tasks to keep them private and free of per-seat fees, then route only the rare heavy job to a paid API.
