Blog · 24 September 2026 · 9 min read
How to set up a local AI in your company: step-by-step guide (2026)
A local AI isn't a research project: with today's open-source tools you can have a ChatGPT-like assistant on a company computer in a few hours. Here are the steps we use ourselves, with the commands and the pitfalls to avoid.
In short: you need three pieces. 1) Ollama, the program that downloads and runs the models (stable version as of 24 September 2026: 0.34.4). 2) An open model that fits the computer's memory, for example Gemma 4 12B (7.6 GB) or Qwen3.8 27B (18 GB). 3) Open WebUI, the chat interface with personal accounts (version 0.11.4). Install them in that order, and make the AI reachable by colleagues only from the internal network or via VPN, never directly from the internet.
What you need before you start
The most useful rule: the whole model must fit in memory. On a Mac with Apple Silicon, unified memory is what counts; on a PC or server with a graphics card, it's the GPU memory (VRAM). As a rough guide, with the 4-bit compressed models that Ollama downloads:
| Available memory | What runs well on it | Example |
|---|---|---|
| 16 GB | Small models, simple tasks, testing | Gemma 4 12B (7.6 GB) |
| 32 GB | Mid-sized models, good quality in Italian | Qwen3.8 27B (18 GB), Gemma 4 26B (19 GB) |
| 64 GB and above | Large models, long documents, more users | gpt-oss 120B (65 GB) |
Always leave some headroom: besides the model, you need memory for the operating system and for the context (the text the AI keeps in mind during a conversation), which grows with long documents and simultaneous users. To choose the model, you'll find a comparison in the guide to models for Italian.
Step 1 – Install Ollama
Ollama downloads the models, loads them into memory and exposes them through a programming interface (API) on port 11434.
- macOS (14 Sonoma or later): download
Ollama.dmgfrom ollama.com/download and drag the app into Applications. On Macs with M-series chips it uses the integrated GPU. - Windows (10 22H2 or later): download and run
OllamaSetup.exefrom the same page. - Linux: the official script installs Ollama as a system service.
curl -fsSL https://ollama.com/install.sh | shTo check it works, open the terminal and type ollama --version.
Step 2 – Download and test a model
Start with a model that fits comfortably in your memory. For example, on a computer with 16 GB:
ollama pull gemma4:12b
ollama run gemma4:12bA chat opens in the terminal: ask a few questions in Italian about your work. Other useful commands:
ollama list: the downloaded models and how much space they take up.ollama ps: the models currently loaded in memory, and whether they are using the GPU.ollama rm nome-modello: removes a model you no longer use.
Tip: test two or three models on the same real questions (an email to write, a document to summarise). It's the quickest way to understand what quality you need, and therefore which hardware.
Step 3 – Install the chat interface (Open WebUI)
Chatting in the terminal is fine for testing, but your colleagues need a ChatGPT-style interface with personal accounts and history. Open WebUI is installed with Docker; with Ollama already running on the same computer, the command from the official documentation is:
docker run -d -p 3000:8080 \
--add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data \
-e WEBUI_SECRET_KEY=la-tua-chiave-segreta \
--name open-webui --restart always \
ghcr.io/open-webui/open-webui:mainGenerate the secret key with openssl rand -hex 32 instead of making one up. Then open http://localhost:3000 in your browser.
- The first account created becomes the administrator: create it yourself straight away, before giving the address to anyone.
- Accounts created afterwards remain pending until an administrator approves them.
- With groups you can decide who sees which models and which document bases.
If you'd rather not use Docker, for personal use there's also LM Studio, a desktop app that's free for internal work use too.
Step 4 – Make it securely accessible to colleagues
This is the step where most mistakes are made. Three rules:
- Never expose Ollama to the internet. Ollama has no authentication: anyone who can reach port 11434 can use your models. In February 2026, LeakIX researchers found around 12,000 Ollama instances reachable from the internet. If Open WebUI runs on the same computer, Ollama can listen locally only (this is the default setting).
- Have colleagues go through Open WebUI, which has accounts and permissions, not through the Ollama API.
- Use a VPN for access from outside the office (for example WireGuard or Tailscale), not port forwarding on the router.
Watch out with Docker on Linux: ports published by Docker can bypass the system firewall rules (such as ufw). If the server has a public address, publish the port only on the internal address, for example -p 127.0.0.1:3000:8080 behind a reverse proxy, or check the rules carefully.
Step 5 – Backups, updates and licences
- Backup: Open WebUI accounts, chats and documents live in the Docker volume
open-webui. Include it in your company backup. - Updates: Ollama and Open WebUI release new versions every few weeks. Update after reading the release notes and with a fresh backup.
- Licences: check each model's licence before commercial use. Gemma 4, Qwen3.8 and gpt-oss use Apache 2.0. Open WebUI has its own licence: above 50 users a month you cannot remove the "Open WebUI" branding without an enterprise licence.
The most common mistakes
- Model too large: if it doesn't fit in memory it becomes painfully slow. A fast mid-sized model is better than a large unusable one.
- Context too short: on GPUs with less than 24 GB, Ollama uses a default context of 4,096 tokens, which cuts off long documents. Increase it in the model settings (the
num_ctxparameter), bearing in mind that it uses memory. - Expecting ChatGPT: local models are excellent for office tasks, but the largest cloud models are still better at very complex reasoning. Choose your use cases accordingly.
- No usage rules: decide beforehand which data may be uploaded and who can see what, and tell your colleagues.
When it's worth getting help
Want it all in a PDF to print or share with colleagues? Download the complete guide for free: hardware, models, installation, security and a 20-point checklist.
A trial on one computer takes an afternoon. Taking it into production for a company is a different job: accounts and permissions per department, secure remote access, backups, updates, connection to management systems and staff training (which also helps with the AI Act).
If the trial has convinced you and you want a reliable installation, get in touch: we'll start from what you've already tested. Dabryx consultancy starts at €100 per hour.
Sources
- Ollama – release v0.34.4 (GitHub, 23 September 2026)
- Ollama – installing on macOS (official documentation)
- Ollama – installing on Linux (official documentation)
- Ollama – FAQ: port 11434 and the OLLAMA_HOST variable
- LeakIX – Ollama exposed: thousands of instances reachable from the internet (February 2026)
- Open WebUI – release v0.11.4 (GitHub, 21 September 2026)
- Open WebUI – Quick start with Docker (official documentation)
- Open WebUI – roles, permissions and groups
- Open WebUI – licence (branding clause)
- Open WebUI – RAG: settings and context limits
- LM Studio – free for work use too (8 July 2025)
- Ollama – Gemma 4 (sizes and variants)
- Ollama – Qwen3.8
Frequently asked questions
How long does it take to set up a local AI?
A trial on one computer takes just a few hours: installing Ollama, downloading a model and Open WebUI. A company installation with accounts, VPN, backups, connected documents and training usually takes a few days of work.
Do you need to know how to code?
Not for the trial: a few commands to copy are enough. For company use you need systems skills (Docker, networking, backups, security), which you can have in-house or entrust to a supplier.
Can I use a computer I already have?
Yes, if it has enough memory for the chosen model. A Mac with an M-series chip and 16 GB runs small models; for higher-quality models in Italian you need 32 GB or more.
Want an estimate for your company?
Tell us how many people will use the AI and what for: we'll tell you which configuration you need.
Response times
Within 2 hours on business days (9:00–18:00) for clients with a support contract.
AI consulting
From €100 per hour, on your company's servers or Macs.
Where we work
All over Italy, remotely and on site.