5 Docker Containers You Need for a Home AI Lab

Ollama is the engine room. It pulls and runs local models and exposes an HTTP endpoint on port 11434. Every other container in the lab talks to that endpoint, so nothing else needs to know how a model is loaded. The official Docker Hub image ships three routes: CPU only, NVIDIA GPUs through the NVIDIA Container Toolkit with --gpus=all, and AMD GPUs through the rocm tag.

The install detail that matters most is the volume. Models are large, and a container recreated without a named volume downloads them all again. Worse, it can lose a model you pulled from a tag that has since moved. The documented run command is:

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Drop --gpus=all for the CPU route. One honest caveat: GPU pass-through depends on the host, not on Ollama. The documented Docker GPU routes are Linux and Windows with WSL2. Docker Desktop on macOS does not hand containers an NVIDIA GPU, so a Mac lab either runs Ollama natively for speed or accepts CPU inference inside Docker.

A terminal is fine for testing a model and useless for living with one. Open WebUI puts a browser interface on the lab: chats, users, file uploads, and connections to Ollama and to any other provider you add. Docker is the officially recommended install, and the quick start is one command:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data -e WEBUI_SECRET_KEY=your-secret-key --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Two of those flags earn their place. The volume at /app/backend/data holds users, chats and uploaded files. Without it, every recreate starts the lab from a blank login page. WEBUI_SECRET_KEY signs sessions, so set it once with something like openssl rand -hex 32 and keep it, because a container that regenerates the key logs everyone out. The :cuda and :ollama image variants exist for GPU acceleration and for bundling Ollama inside the same container. Both are options rather than the default, since a separate Ollama container is easier to upgrade and to share between tools.

Do you need a vector database at all? For casual chat, no, and skipping it is a legitimate decision. The moment the lab is supposed to answer questions from your own documents, notes or project files, those files have to be embedded and searched. That is what Qdrant does: it stores vectors and finds the nearest ones. It is the memory the AI consults, which is a different thing from the AI itself.

The local quick start runs Qdrant with two ports, 6333 for the REST API and dashboard and 6334 for gRPC. The quick start also mounts a storage path so collections survive restarts. The dashboard at localhost:6333/dashboard is worth opening once, because seeing a collection fill up is the fastest way to confirm an ingestion pipeline is doing anything. A model alone is not a useful system, a point the earlier KDnuggets piece on self-hosted LLMs in the real world makes at length. Retrieval is usually the layer that turns a demo into something you reach for.

Every lab eventually needs glue. Files land in a folder and need embedding, model endpoints need calling on a schedule, and summaries need pushing somewhere a human will read them. n8n is a workflow tool that runs those jobs visually. Its Docker Compose install is the sensible route for a lab, because Compose is how the rest of the stack should be described anyway. It serves on localhost:5678 and defaults to SQLite, with Postgres as the option once workflows multiply.