AI Local Toolkit

A self-hosted AI stack bundling an LLM gateway, workflow automation, and a chat UI — all backed by a shared PostgreSQL database. Deployable locally with Docker Compose or on any Kubernetes cluster via Helm.

LiteLLM ships with no models — register the providers you want in the LiteLLM admin UI, then use them from Open WebUI.

⚠️ Warning: This project is not production-ready out of the box. It has no TLS, no authentication proxy, no secrets management, and no backup strategy. It is safe for local use and trusted internal networks. See Production Checklist before exposing it to the internet or running it with real data.

Services

Architecture

┌─────────────┐              ┌─────────────┐
│  Open WebUI │              │     n8n     │
└──────┬──────┘              └──────┬──┬───┘
       │                            │  │
       │    OpenAI-compatible API   │  │ workflow storage
       └─────────────┬──────────────┘  │
                     ▼                 ▼
              ┌────────────┐    ┌────────────┐
              │  LiteLLM   │───▶│ PostgreSQL │
              └────────────┘    └────────────┘

Services communicate internally by container/pod name.

Open WebUI ↔ LiteLLM

Open WebUI is wired to LiteLLM over its OpenAI-compatible API. The stack injects only the host — never an API key. On first login, open the LiteLLM UI at http://localhost:4000/ui, create a virtual key, and paste it into Open WebUI under Settings → Connections. The LITELLM_MASTER_KEY is intentionally never given to Open WebUI — use a scoped virtual key instead.

Prerequisites

• Docker Compose: Docker Engine + Docker Compose plugin
• Kubernetes: a running cluster + Helm 3

Deployment

Option A — Docker Compose (local)

1. Create the shared network (one-time)

docker network create ai-toolkit

2. Configure environment

cp .env.dist .env

Edit .env:

POSTGRES_DB=postgres
POSTGRES_USER=postgres
POSTGRES_PASSWORD=<your_secure_password>

LITELLM_MASTER_KEY=sk-<generate_a_key>
LITELLM_SALT_KEY=sk-<generate_a_key>

TZ=America/New_York   # IANA timezone

3. Start the stack

docker compose up -d

4. Verify

docker compose ps   # wait until all services show "healthy"

Disabling services (optional)

Edit COMPOSE_PROFILES in .env to remove services you don't need:

# All enabled (default)
COMPOSE_PROFILES=litellm,n8n,openwebui

# Without n8n
COMPOSE_PROFILES=litellm,openwebui

# LiteLLM only (no UI, no automation)
COMPOSE_PROFILES=litellm

Postgres always runs — it is not profile-gated since other services depend on it. Disabling litellm requires also removing openwebui (it depends on LiteLLM).

Option B — Kubernetes / Helm

1. Install the chart

helm install ai-toolkit ./helm/ai-toolkit \
  --namespace ai-toolkit --create-namespace \
  --set litellm.masterKey=sk-... \
  --set litellm.saltKey=sk-... \
  --set postgresql.auth.password=...

For repeatable installs, use a values file instead of --set flags:

# my-values.yaml
litellm:
  masterKey: sk-...
  saltKey: sk-...

postgresql:
  auth:
    password: ...

helm install ai-toolkit ./helm/ai-toolkit \
  --namespace ai-toolkit --create-namespace \
  -f my-values.yaml

2. Access services (without Ingress)

kubectl port-forward svc/ai-toolkit-litellm 4000:4000
kubectl port-forward svc/ai-toolkit-n8n 5678:5678
kubectl port-forward svc/ai-toolkit-openwebui 3000:8080

3. Enable Ingress (optional)

Add to your values file:

ingress:
  enabled: true
  className: nginx
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
  hosts:
    litellm: litellm.example.com
    n8n: n8n.example.com
    openwebui: chat.example.com
  tls:
    - secretName: ai-toolkit-tls
      hosts:
        - litellm.example.com
        - n8n.example.com
        - chat.example.com

4. Use an external database (optional)

To disable the bundled PostgreSQL and point to your own (e.g. RDS, Cloud SQL):

postgresql:
  enabled: false

externalDatabase:
  host: my-db.example.com
  port: 5432
  username: postgres
  password: ...
  database: postgres

The external server must already have n8n and litellm databases created.

5. Disable individual services (optional)

n8n:
  enabled: false   # skip n8n if you don't need workflow automation

openwebui:
  enabled: false   # skip Open WebUI if you use a different frontend

Upgrade / uninstall

helm upgrade ai-toolkit ./helm/ai-toolkit -f my-values.yaml
helm uninstall ai-toolkit

Project Structure

.
├── docker-compose.yml            # Docker Compose stack definition
├── .env.dist                     # Environment variables template
├── config/
│   ├── litellm_config.yaml       # LiteLLM model registry (empty by default)
│   └── init_db.sh                # Creates n8n & litellm databases on first run
├── helm/
│   └── ai-toolkit/               # Helm chart for Kubernetes
│       ├── Chart.yaml
│       ├── values.yaml           # All chart defaults, fully annotated
│       └── templates/            # K8s manifests (Deployments, Services, PVCs, …)
└── .github/
    └── workflows/
        └── validate.yml          # CI: validates compose file and Helm chart

Models

LiteLLM ships with no models pre-configured — register the providers you want yourself. Any OpenAI-compatible or LiteLLM-supported provider works (OpenAI, Anthropic, Vertex, Groq, a local model server, …). See Adding models to LiteLLM below.

Configuration

Adding models to LiteLLM

LiteLLM ships with an empty model registry. Add models in config/litellm_config.yaml (Docker Compose) or via litellm.extraModels in your Helm values:

- model_name: gpt-4o
  litellm_params:
    model: openai/gpt-4o
    api_key: os.environ/OPENAI_API_KEY

Pair any provider key with litellm.extraEnv (Helm) or the litellm service environment (Docker Compose). You can also add models at runtime in the LiteLLM UI — STORE_MODEL_IN_DB persists them to Postgres.

Docker Compose: docker compose restart litellm Kubernetes: helm upgrade ai-toolkit ./helm/ai-toolkit -f my-values.yaml

Calling the LiteLLM API directly

curl http://localhost:4000/v1/models \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY"

PostgreSQL databases

A single PostgreSQL 16 instance hosts two databases: n8n and litellm. The config/init_db.sh script creates them on first startup and is idempotent (safe to re-run).

To reset the database (Docker Compose):

docker compose down
docker volume rm toolkit_postgres-data
docker compose up -d

Docker Compose — Useful Commands

docker compose up -d                    # start all services
docker compose down                     # stop all services
docker compose ps                       # show service health
docker compose logs -f litellm          # tail logs for one service
docker compose restart litellm          # reload after config change
docker compose down -v && docker compose up -d  # full reset (deletes data)

Troubleshooting

docker compose up fails with "network ai-toolkit not found"

docker network create ai-toolkit

Services stay starting or unhealthy LiteLLM and n8n wait for Postgres to pass its healthcheck before starting. On first run, Postgres runs the init script which can take 20–30 s. Check progress:

docker compose logs -f postgres

Open WebUI shows no models Open WebUI ships with the LiteLLM host wired in but no API key — add a LiteLLM virtual key (from http://localhost:4000/ui) under Settings → Connections. LiteLLM itself serves no models until you register them (see Adding models to LiteLLM above):

docker compose logs -f litellm

n8n can't connect to the database The POSTGRES_USER and POSTGRES_PASSWORD in .env must match what was used when the postgres-data volume was first created. If they changed, reset the volume or revert the credentials.

Port already in use Change the host-side port in docker-compose.yml, e.g. "4001:4000" for LiteLLM.

Helm pod stuck in Pending The cluster likely has no default StorageClass for PVCs. Either install one or set persistence.storageClass in your values to an available class:

kubectl get storageclass

Production Checklist

This stack ships with sane defaults for development and internal use. The items below are required before running it with real users or sensitive data.

Must-have

Should-have