LLM Harbor
https://github.com/user-attachments/assets/e4897391-c5a8-4391-93c3-9f8b76155f11 [](https://github.com/av/harbor/releases)    [](https://visitorbadge.io/status?path=av%2Fharbor) [](https://discord.gg/8nDRphrhSF)  [](https://textclip.sh?ask=claude#c=TZTNjiM1EIDveQofga2N4LJCkQZpZ2AAaVZCzBNU29XdtSm7PLY7P2QjjYQUshyjleC6J8S8Bnd4h34S5E4GcbH6p7r8lb-q_g5To2l8__Hm7vvx-PQ6xsXXapeUbtRHzTQeTqIW5e7ujRni-OvPa2oGfqEi6PEFpm41m_39YfETJX1pNbTcjcfHNScOnfnCejceTu0gkgvapcGhTFGBbBmPj7truIX7vYlJWxa6X2M09La-O5xyweBQNJApQwgk4-HEoVAKVMxDGg8nrw0LmfH4p0Xbk_nn8ZOK86npORfz12_2XIB5dz0en27H49P9u69efT6bXV_tzvQwrTZGWIl4KB1DwabZYmTA2Cd1XAg855JQUobcCYYOliVhyK0mTymDeEdRdAvIqWYJtEFYaqPiwKsbBJPHDeRIFTJDxCSU9rdXu-kgQbhJZHssUJf6QJvzfX-hrF9MFw1rYNvFAhi2pefQ1Q1rbGAFz8shogMtlDxY9e124P391S4Tpk3oIFKKQhu2CI7bLUTB4GgDwoVqolpcO2QCjRwK2NaB9a6HFpvEFpAdJUCe2KrICiKeVijwdojbQgk0UugxuFyTyrawzZAoakS7hPBlgEalQORIwoEyPLiEocDyFcSkPpZWFdbUFI2gPnDElClBK7rmTGdA0TVoLFyJvabYs4X8IBNVF4snKAmp5SUIFgolW40ECbta5tl4XmM8n_tkssaBp4LeRvA2KohLMDU9MmBHodTuroLXhCsCp1Y4dNAkXWdK9ciGkEVLD2sOzrMINBRsD41qLvvZbFI9_vLh2lxkjIfTxf_F-nMp_5MkLu0XP77-1jz3zmKlbMlc1C7Yd-aCPR7_qOCLNzc_jIfTlHg22w0RnK4DxCohF0wFRLsMtCELuaeKObC4SRsMSeAhwXnY9tmch3m-E-gg75er5ymd7zIMIHuT5zuvjgQwdXlv-na-azk4cALZYtibc__OP_tvprdezsN5_i-Mx99ns282UZCDKT3nukUNNUWNp_m_) [](https://textclip.sh?ask=chatgpt#c=TZTNjiM1EIDveQofga2N4LJCkQZpZ2AAaVZCzBNU29XdtSm7PLY7P2QjjYQUshyjleC6J8S8Bnd4h34S5E4GcbH6p7r8lb-q_g5To2l8__Hm7vvx-PQ6xsXXapeUbtRHzTQeTqIW5e7ujRni-OvPa2oGfqEi6PEFpm41m_39YfETJX1pNbTcjcfHNScOnfnCejceTu0gkgvapcGhTFGBbBmPj7truIX7vYlJWxa6X2M09La-O5xyweBQNJApQwgk4-HEoVAKVMxDGg8nrw0LmfH4p0Xbk_nn8ZOK86npORfz12_2XIB5dz0en27H49P9u69efT6bXV_tzvQwrTZGWIl4KB1DwabZYmTA2Cd1XAg855JQUobcCYYOliVhyK0mTymDeEdRdAvIqWYJtEFYaqPiwKsbBJPHDeRIFTJDxCSU9rdXu-kgQbhJZHssUJf6QJvzfX-hrF9MFw1rYNvFAhi2pefQ1Q1rbGAFz8shogMtlDxY9e124P391S4Tpk3oIFKKQhu2CI7bLUTB4GgDwoVqolpcO2QCjRwK2NaB9a6HFpvEFpAdJUCe2KrICiKeVijwdojbQgk0UugxuFyTyrawzZAoakS7hPBlgEalQORIwoEyPLiEocDyFcSkPpZWFdbUFI2gPnDElClBK7rmTGdA0TVoLFyJvabYs4X8IBNVF4snKAmp5SUIFgolW40ECbta5tl4XmM8n_tkssaBp4LeRvA2KohLMDU9MmBHodTuroLXhCsCp1Y4dNAkXWdK9ciGkEVLD2sOzrMINBRsD41qLvvZbFI9_vLh2lxkjIfTxf_F-nMp_5MkLu0XP77-1jz3zmKlbMlc1C7Yd-aCPR7_qOCLNzc_jIfTlHg22w0RnK4DxCohF0wFRLsMtCELuaeKObC4SRsMSeAhwXnY9tmch3m-E-gg75er5ymd7zIMIHuT5zuvjgQwdXlv-na-azk4cALZYtibc__OP_tvprdezsN5_i-Mx99ns282UZCDKT3nukUNNUWNp_m_) [](https://textclip.sh?ask=perplexity#c=TZTNjiM1EIDveQofga2N4LJCkQZpZ2AAaVZCzBNU29XdtSm7PLY7P2QjjYQUshyjleC6J8S8Bnd4h34S5E4GcbH6p7r8lb-q_g5To2l8__Hm7vvx-PQ6xsXXapeUbtRHzTQeTqIW5e7ujRni-OvPa2oGfqEi6PEFpm41m_39YfETJX1pNbTcjcfHNScOnfnCejceTu0gkgvapcGhTFGBbBmPj7truIX7vYlJWxa6X2M09La-O5xyweBQNJApQwgk4-HEoVAKVMxDGg8nrw0LmfH4p0Xbk_nn8ZOK86npORfz12_2XIB5dz0en27H49P9u69efT6bXV_tzvQwrTZGWIl4KB1DwabZYmTA2Cd1XAg855JQUobcCYYOliVhyK0mTymDeEdRdAvIqWYJtEFYaqPiwKsbBJPHDeRIFTJDxCSU9rdXu-kgQbhJZHssUJf6QJvzfX-hrF9MFw1rYNvFAhi2pefQ1Q1rbGAFz8shogMtlDxY9e124P391S4Tpk3oIFKKQhu2CI7bLUTB4GgDwoVqolpcO2QCjRwK2NaB9a6HFpvEFpAdJUCe2KrICiKeVijwdojbQgk0UugxuFyTyrawzZAoakS7hPBlgEalQORIwoEyPLiEocDyFcSkPpZWFdbUFI2gPnDElClBK7rmTGdA0TVoLFyJvabYs4X8IBNVF4snKAmp5SUIFgolW40ECbta5tl4XmM8n_tkssaBp4LeRvA2KohLMDU9MmBHodTuroLXhCsCp1Y4dNAkXWdK9ciGkEVLD2sOzrMINBRsD41qLvvZbFI9_vLh2lxkjIfTxf_F-nMp_5MkLu0XP77-1jz3zmKlbMlc1C7Yd-aCPR7_qOCLNzc_jIfTlHg22w0RnK4DxCohF0wFRLsMtCELuaeKObC4SRsMSeAhwXnY9tmch3m-E-gg75er5ymd7zIMIHuT5zuvjgQwdXlv-na-azk4cALZYtibc__OP_tvprdezsN5_i-Mx99ns282UZCDKT3nukUNNUWNp_m_) [](https://textclip.sh?failmsg=It%20appears%20that%20you%20do%20not%20have%20Harbor%20installed%2C%20or%20Open%20WebUI%20is%20not%20running%20on%20default%20port&redirect=http%3A%2F%2Flocalhost%3A33801%3Fq%3D__TEXT__#c=TZTNjiM1EIDveQofga2N4LJCkQZpZ2AAaVZCzBNU29XdtSm7PLY7P2QjjYQUshyjleC6J8S8Bnd4h34S5E4GcbH6p7r8lb-q_g5To2l8__Hm7vvx-PQ6xsXXapeUbtRHzTQeTqIW5e7ujRni-OvPa2oGfqEi6PEFpm41m_39YfETJX1pNbTcjcfHNScOnfnCejceTu0gkgvapcGhTFGBbBmPj7truIX7vYlJWxa6X2M09La-O5xyweBQNJApQwgk4-HEoVAKVMxDGg8nrw0LmfH4p0Xbk_nn8ZOK86npORfz12_2XIB5dz0en27H49P9u69efT6bXV_tzvQwrTZGWIl4KB1DwabZYmTA2Cd1XAg855JQUobcCYYOliVhyK0mTymDeEdRdAvIqWYJtEFYaqPiwKsbBJPHDeRIFTJDxCSU9rdXu-kgQbhJZHssUJf6QJvzfX-hrF9MFw1rYNvFAhi2pefQ1Q1rbGAFz8shogMtlDxY9e124P391S4Tpk3oIFKKQhu2CI7bLUTB4GgDwoVqolpcO2QCjRwK2NaB9a6HFpvEFpAdJUCe2KrICiKeVijwdojbQgk0UugxuFyTyrawzZAoakS7hPBlgEalQORIwoEyPLiEocDyFcSkPpZWFdbUFI2gPnDElClBK7rmTGdA0TVoLFyJvabYs4X8IBNVF4snKAmp5SUIFgolW40ECbta5tl4XmM8n_tkssaBp4LeRvA2KohLMDU9MmBHodTuroLXhCsCp1Y4dNAkXWdK9ciGkEVLD2sOzrMINBRsD41qLvvZbFI9_vLh2lxkjIfTxf_F-nMp_5MkLu0XP77-1jz3zmKlbMlc1C7Yd-aCPR7_qOCLNzc_jIfTlHg22w0RnK4DxCohF0wFRLsMtCELuaeKObC4SRsMSeAhwXnY9tmch3m-E-gg75er5ymd7zIMIHuT5zuvjgQwdXlv-na-azk4cALZYtibc__OP_tvprdezsN5_i-Mx99ns282UZCDKT3nukUNNUWNp_m_) Setup your local LLM stack effortlessly. ```bash # Starts fully configured Open WebUI and Ollama harbor up # Now, Open WebUI can do Web RAG and TTS/STT harbor up searxng speaches ``` Harbor is a CLI and companion app that lets you spin up a complete local LLM stack—backends like Ollama, llama.cpp, or vLLM, frontends like Open WebUI, plus supporting services like SearXNG for web search, Speaches for voice chat, and ComfyUI for image generation—all pre-wired to work together with a single `harbor up` command. No manual setup: just pick the services you want and Harbor handles the Docker Compose orchestration, configuration, and cross-service connectivity so you can focus on actually using your models.  ## News - **v0.4.19** - Boost Anthropic and Responses API compat layers, `harbor launch` command, ik_llama.cpp service, Boost workflows and tools module - **v0.4.18** - Open Design and Voicebox services, with local design workflows, voice generation, Harbor backend integrations, and `harbor how` backed by mi - **v0.4.17** - Needle and npcsh services, with OpenAI-compatible tool-calling, CLI launch support, and npcsh backend/frontend integrations - **v0.4.16** - ML Intern, facts, and mi services, with smarter ML Intern llama.cpp model selection and local backend integrations - **v0.4.15** - Lemonade and Tokscale services, TabbyAPI official image and model fixes, profile security hardening - **v0.4.14** - Bifrost AI gateway and SurfSense research workspace, Harbor CLI sidecar integrations, and Unsloth Studio portability fixes - **v0.4.13** - Unsloth Studio service with zero-click API key bootstrap and six cross-integrations, workspace-init sidecar pattern rolled out across 17 services for host-user-owned bind mounts ## Documentation - [Installing Harbor](https://github.com/av/harbor/wiki/1.0.-Installing-Harbor)<br/> Guides to install Harbor CLI and App, including the [agent install prompt](./install.md) for Claude Code, Codex, and similar tools - [Harbor User Guide](https://github.com/av/harbor/wiki/1.-Harbor-User-Guide)<br/> High-level overview of working with Harbor - [Harbor App](https://github.com/av/harbor/wiki/1.1-Harbor-App)<br/> Overview and manual for the Harbor companion application - [Harbor Services](https://github.com/av/harbor/wiki/2.-Services)<br/> Catalog of services available in Harbor - [Harbor CLI Reference](https://github.com/av/harbor/wiki/3.-Harbor-CLI-Reference)<br/> Read more about Harbor CLI commands and options. Read about supported services and the ways to configure them. - [Harbor Guides](https://github.com/av/harbor/wiki/8.-Guides)<br/> Practical guides for local LLM stack setup, local web RAG, coding agents, OpenAI-compatible backends, and self-hosted AI stack planning. - [Join our Discord](https://discord.gg/8nDRphrhSF)<br/> Get help, share your experience, and contribute to the project. ## What can Harbor do?  #### ✦ Local LLMs Run LLMs and related services locally, with no or minimal configuration, typically in a single command or click. ```bash # All backends are pre-connected to Open WebUI harbor up ollama harbor up llamacpp harbor up vllm # Host-native Metal/GPU inference on macOS harbor up dmr harbor up mlx harbor up omlx # Set and remember args for llama.cpp harbor llamacpp args -ngl 32 ``` #### Local Coding Tools Use Harbor backends from installed coding and agent CLIs without hand-editing each tool's provider config. [`harbor launch`](./docs/3.-Harbor-CLI-Reference.md#harbor-launch-launch-options---service-servicetool-args) starts or detects a Harbor OpenAI-compatible backend, wires the selected model into the host tool, and leaves the tool running in your current project directory. ```bash # Run Codex against an explicit Harbor backend and model harbor launch --backend ollama --model qwen3.5:4b codex # Add Harbor Boost web search and URL-reading tools harbor launch --web --backend ollama --model qwen3.5:4b codex # Write adapter config without starting the tool harbor launch --config opencode # Force the container service when a name also has a host adapter harbor launch --service opencode --help ``` Supported host tools include `claude`, `codex`, `copilot`, `droid`, `hermes`, `mi`, `openclaw`, `opencode`, `pi`, `pool`, and `vscode`. #### Cutting Edge Inference Harbor supports most of the major inference engines as well as a few of the lesser-known ones. On macOS, [Docker Model Runner](./docs/2.2.22-Backend-Docker-Model-Runner.md), [MLX](./docs/2.2.23-Backend-MLX.md), and [oMLX](./docs/2.2.24-Backend-oMLX.md) run inference on the host with Metal acceleration — no containers needed for the compute. ```bash # We sincerely hope you'll never try to run all of them at once harbor up vllm llamacpp tgi litellm tabbyapi aphrodite sglang ktransformers mistralrs airllm # Host-native inference (macOS Metal / Linux GPU) harbor up dmr mlx omlx ``` #### Tool Use Enjoy the benefits of MCP ecosystem, extend it to your use-cases. ```bash # Manage MCPs with a convenient Web UI harbor up metamcp # Connect MCPs to Open WebUI harbor up metamcp mcpo ``` #### Generate Images Harbor includes ComfyUI + Flux + Open WebUI integration. ```bash # Use FLUX in Open WebUI in one command harbor up comfyui ``` #### Local Web RAG / Deep Research Harbor includes [SearXNG](./docs/2.3.1-Satellite&colon-SearXNG.md) that is pre-connected to a lot of services out of the box: [Perplexica](./docs/2.3.2-Satellite&colon-Perplexica.md), [ChatUI](./docs/2.1.4-Frontend&colon-ChatUI.md), [Morphic](./docs/2.3.34-Satellite-Morphic.md), [Local Deep Research](./docs/2.3.45-Satellite-Local-Deep-Research.md) and more. ```bash # SearXNG is pre-connected to Open WebUI harbor up searxng # And to many other services harbor up searxng chatui harbor up searxng morphic harbor up searxng perplexica harbor up searxng ldr ``` #### LLM Workflows Harbor includes multiple services for build LLM-based data and chat workflows: [Dify](./docs/2.3.3-Satellite&colon-Dify.md), [LitLytics](./docs/2.3.21-Satellite&colon-LitLytics.md), [n8n](./docs/2.3.23-Satellite&colon-n8n.md), [Open WebUI Pipelines](./docs/2.3.25-Satellite&colon-Open-WebUI-Pipelines.md), [FloWise](./docs/2.3.31-Satellite&colon-Flowise.md), [LangFlow](./docs/2.3.32-Satellite&colon-LangFlow.md) ```bash # Use Dify in Open WebUI harbor up dify ``` #### Talk to your LLM Setup voice chats with your LLM in a single command. Open WebUI + Speaches ```bash # Speaches includes OpenAI-compatible SST and TTS # and connected to Open WebUI out of the box harbor up speaches ``` #### Chat from the phone You can access Harbor services from your phone with a QR code. Easily get links for local, LAN or Docker access. ```bash # Print a QR code to open the service on your phone harbor qr # Print a link to open the service on your phone harbor url webui ``` #### Chat from anywhere Harbor includes a [built-in tunneling service](./docs/3.-Harbor-CLI-Reference.md#harbor-tunnel-service) to expose your Harbor to the internet. > [!WARN] > Be careful exposing your computer to the Internet, it's not safe. ```bash # Expose default UI to the internet harbor tunnel # Expose a specific service to the internet # ⚠️ Ensure to configure authentication for the service harbor tunnel vllm # Harbor comes with traefik built-in and pre-configured # for all included services harbor up traefik ``` #### LLM Scripting [Harbor Boost](./docs/5.2.-Harbor-Boost.md) allows you to [easily script workflows](./docs/5.2.1.-Harbor-Boost-Custom-Modules.md) and interactions with downstream LLMs. ```bash # Use Harbor Boost to script LLM workflows harbor up boost ``` #### Config Profiles Save and manage configuration profiles for different scenarios. For example - save [llama.cpp](./docs/2.2.2-Backend&colon-llama.cpp.md) args for different models and contexts and switch between them easily. ```bash # Save and use config profiles harbor profile save llama4 harbor profile use default # Import profiles from a URL harbor profile use https://example.com/path/to/harbor-profile.env ``` #### Command History Harbor keeps a [local-only history of recent commands](./docs/3.-Harbor-CLI-Reference.md#harbor-history). Look up and re-run easily, standalone from the system shell history. ```bash # Lookup recently used harbor commands harbor history ``` #### Eject Ready to move to your own setup? Harbor [will give you](./docs/3.-Harbor-CLI-Reference.md#harbor-eject) a docker-compose file replicating your setup. ```bash # Eject from Harbor into a standalone Docker Compose setup # Will export related services and variables into a standalone file. harbor eject searxng llamacpp > docker-compose.harbor.yml ``` --- ## Services ##### UIs [AnythingLLM](https://github.com/av/harbor/wiki/2.1.9-Frontend:-AnythingLLM) ⦁︎ [AstrBot](https://github.com/av/harbor/wiki/2.3.63-Satellite-AstrBot) ⦁︎ [BionicGPT](https://github.com/av/harbor/wiki/2.1.8-Frontend:-BionicGPT) ⦁︎ [Chat Nio](https://github.com/av/harbor/wiki/2.1.10-Frontend:-Chat-Nio) [ComfyUI](https://github.com/av/harbor/wiki/2.1.2-Frontend:-ComfyUI) ⦁︎ [Hollama](https://github.com/av/harbor/wiki/2.1.6-Frontend:-hollama) ⦁︎ [HuggingFace ChatUI](https://github.com/av/harbor/wiki/2.1.4-Frontend:-ChatUI) ⦁︎ [KoboldCpp](https://github.com/av/harbor/wiki/2.2.16-Backend:-KoboldCpp) [LibreChat](https://github.com/av/harbor/wiki/2.1.3-Frontend:-LibreChat) ⦁︎ [Lobe Chat](https://github.com/av/harbor/wiki/2.1.5-Frontend:-Lobe-Chat) ⦁︎ [Mikupad](https://github.com/av/harbor/wiki/2.1.11-Frontend:-Mikupad) ⦁︎ [ol1](https://github.com/av/harbor/wiki/2.3.19-Satellite:-ol1) [Omnichain](https://github.com/av/harbor/wiki/2.3.16-Satellite:-omnichain) ⦁︎ [Onyx](https://github.com/av/harbor/wiki/2.1.14-Frontend-Onyx) ⦁︎ [Open WebUI](https://github.com/av/harbor/wiki/2.1.1-Frontend:-Open-WebUI) ⦁︎ [oterm](https://github.com/av/harbor/wiki/2.1.12-Frontend-oterm) [Parllama](https://github.com/av/harbor/wiki/2.1.7-Frontend:-parllama) ⦁︎ [SillyTavern](https://github.com/av/harbor/wiki/2.1.15-Frontend-SillyTavern) ⦁︎ [Voicebox](https://github.com/av/harbor/wiki/2.1.16-Frontend-Voicebox) ##### Backends [AirLLM](https://github.com/av/harbor/wiki/2.2.11-Backend:-AirLLM) ⦁︎ [Aphrodite](https://github.com/av/harbor/wiki/2.2.5-Backend:-Aphrodite-Engine) ⦁︎ [faster-whisper-server](https://github.com/av/harbor/wiki/2.2.14-Backend:-Speaches) ⦁︎ [ik_llama.cpp](https://github.com/av/harbor/wiki/2.2.21-Backend-ik_llama.cpp) [KoboldCpp](https://github.com/av/harbor/wiki/2.2.16-Backend:-KoboldCpp) ⦁︎ [KTransformers](https://github.com/av/harbor/wiki/2.2.13-Backend:-KTransformers) ⦁︎ [Lemonade](https://github.com/av/harbor/wiki/2.2.19-Backend-Lemonade) ⦁︎ [llama.cpp](https://github.com/av/harbor/wiki/2.2.2-Backend:-llama.cpp) [lmdeploy](https://github.com/av/harbor/wiki/2.2.10-Backend:-lmdeploy) ⦁︎ [mistral.rs](https://github.com/av/harbor/wiki/2.2.6-Backend:-mistral.rs) ⦁︎ [Modular MAX](https://github.com/av/harbor/wiki/2.2.17-Backend-Modular-MAX) ⦁︎ [Needle](https://github.com/av/harbor/wiki/2.2.20-Backend-Needle) [Nexa SDK](https://github.com/av/harbor/wiki/2.2.15-Backend:-Nexa-SDK) ⦁︎ [Ollama](https://github.com/av/harbor/wiki/2.2.1-Backend:-Ollama) ⦁︎ [openedai-speech](https://github.com/av/harbor/wiki/2.2.7-Backend:-openedai-speech) ⦁︎ [Parler](https://github.com/av/harbor/wiki/2.2.8-Backend:-Parler) [Docker Model Runner](https://github.com/av/harbor/wiki/2.2.22-Backend-Docker-Model-Runner) ⦁︎ [MLX](https://github.com/av/harbor/wiki/2.2.23-Backend-MLX) ⦁︎ [oMLX](https://github.com/av/harbor/wiki/2.2.24-Backend-oMLX) ⦁︎ [SGLang](https://github.com/av/harbor/wiki/2.2.12-Backend:-SGLang) ⦁︎ [Speaches](https://github.com/av/harbor/wiki/2.2.14-Backend:-Speaches) [TabbyAPI](https://github.com/av/harbor/wiki/2.2.4-Backend:-TabbyAPI) ⦁︎ [Text Generation Inference](https://github.com/av/harbor/wiki/2.2.9-Backend:-text-generation-inference) ⦁︎ [vLLM](https://github.com/av/harbor/wiki/2.2.3-Backend:-vLLM) ##### Satellites [Activepieces](https://github.com/av/harbor/wiki/2.3.64-Satellite-Activepieces) ⦁︎ [Agent Zero](https://github.com/av/harbor/wiki/2.3.47-Satellite-Agent-Zero) ⦁︎ [aichat](https://github.com/av/harbor/wiki/2.3.14-Satellite:-aichat) ⦁︎ [Aider](https://github.com/av/harbor/wiki/2.3.13-Satellite:-aider) [Airweave](https://github.com/av/harbor/wiki/2.3.48-Satellite-Airweave) ⦁︎ [AstrBot](https://github.com/av/harbor/wiki/2.3.63-Satellite-AstrBot) ⦁︎ [autogpt](https://github.com/av/harbor/wiki/2.3.15-Satellite:-AutoGPT) ⦁︎ [Beszel](https://github.com/av/harbor/wiki/2.3.82-Satellite-Beszel) [Bifrost](https://github.com/av/harbor/wiki/2.3.84-Satellite-Bifrost) ⦁︎ [Bolt.new](https://github.com/av/harbor/wiki/2.3.24-Satellite:-Bolt.new) ⦁︎ [Browser Use](https://github.com/av/harbor/wiki/2.3.50-Satellite-Browser-Use) ⦁︎ [cloudflared](https://github.com/av/harbor/wiki/2.3.8-Satellite:-cloudflared) [cmdh](https://github.com/av/harbor/wiki/2.3.9-Satellite:-cmdh) ⦁︎ [Cognee](https://github.com/av/harbor/wiki/2.3.74-Satellite-Cognee) ⦁︎ [DBHub](https://github.com/av/harbor/wiki/2.3.81-Satellite-DBHub) ⦁︎ [DeerFlow](https://github.com/av/harbor/wiki/2.3.65-Satellite-DeerFlow) [Dify](https://github.com/av/harbor/wiki/2.3.3-Satellite:-Dify) ⦁︎ [Docling](https://github.com/av/harbor/wiki/2.3.49-Satellite-Docling) ⦁︎ [Drawio](https://github.com/av/harbor/wiki/2.3.56-Satellite-Drawio) ⦁︎ [Fabric](https://github.com/av/harbor/wiki/2.3.10-Satellite:-fabric) [facts](https://github.com/av/harbor/wiki/2.3.88-Satellite-Facts) ⦁︎ [Flowise](https://github.com/av/harbor/wiki/2.3.31-Satellite:-Flowise) ⦁︎ [gptme](https://github.com/av/harbor/wiki/2.3.36-Satellite-gptme) ⦁︎ [Harbor Bench](https://github.com/av/harbor/wiki/5.1.-Harbor-Bench) [Harbor Boost](https://github.com/av/harbor/wiki/5.2.-Harbor-Boost) ⦁︎ [Hermes Agent](https://github.com/av/harbor/wiki/2.3.76-Satellite-Hermes-Agent) ⦁︎ [Home Assistant](https://github.com/av/harbor/wiki/2.3.60-Satellite-Home-Assistant) ⦁︎ [JupyterLab](https://github.com/av/harbor/wiki/2.3.18-Satellite:-JupyterLab) [K6](https://github.com/av/harbor/wiki/2.3.27-Satellite:-K6) ⦁︎ [Karakeep](https://github.com/av/harbor/wiki/2.3.53-Satellite-Karakeep) ⦁︎ [Khoj](https://github.com/av/harbor/wiki/2.3.67-Satellite-Khoj) ⦁︎ [KoboldCpp](https://github.com/av/harbor/wiki/2.2.16-Backend:-KoboldCpp) [Kotaemon](https://github.com/av/harbor/wiki/2.3.61-Satellite-Kotaemon) ⦁︎ [LangFlow](https://github.com/av/harbor/wiki/2.3.32-Satellite:-LangFlow) ⦁︎ [LangFuse](https://github.com/av/harbor/wiki/2.3.6-Satellite:-langfuse) ⦁︎ [Latent Scope](https://github.com/av/harbor/wiki/2.3.38-Satellite-Latent-Scope) [LibreTranslate](https://github.com/av/harbor/wiki/2.3.41-Satellite-libretranslate) ⦁︎ [LiteLLM](https://github.com/av/harbor/wiki/2.3.5-Satellite:-LiteLLM) ⦁︎ [LitLytics](https://github.com/av/harbor/wiki/2.3.21-Satellite:-LitLytics) ⦁︎ [llama-swap](https://github.com/av/harbor/wiki/2.3.40-Satellite-llamaswap) [lm-evaluation-harness](https://github.com/av/harbor/wiki/2.3.17-Satellite:-lm-evaluation-harness) ⦁︎ [Local Deep Research](https://github.com/av/harbor/wiki/2.3.45-Satellite-Local-Deep-Research) ⦁︎ [LocalAI](https://github.com/av/harbor/wiki/2.3.46-Satellite-LocalAI) ⦁︎ [MCP Forge](https://github.com/av/harbor/wiki/2.3.62-Satellite-MCP-Forge) [mcpo](https://github.com/av/harbor/wiki/2.3.43-Satellite-mcpo) ⦁︎ [MetaMCP](https://github.com/av/harbor/wiki/2.3.42-Satellite-metamcp) ⦁︎ [mi](https://github.com/av/harbor/wiki/2.3.89-Satellite-mi) ⦁︎ [MindsDB](https://github.com/av/harbor/wiki/2.3.57-Satellite-MindsDB) [ML Intern](https://github.com/av/harbor/wiki/2.3.87-Satellite-ML-Intern) ⦁︎ [Morphic](https://github.com/av/harbor/wiki/2.3.34-Satellite-Morphic) ⦁︎ [n8n](https://github.com/av/harbor/wiki/2.3.23-Satellite:-n8n) ⦁︎ [nanobot](https://github.com/av/harbor/wiki/2.3.71-Satellite-nanobot) [Netdata](https://github.com/av/harbor/wiki/2.3.54-Satellite-Netdata) ⦁︎ [npcsh](https://github.com/av/harbor/wiki/2.3.90-Satellite-npcsh) ⦁︎ [OmniParser](https://github.com/av/harbor/wiki/2.3.30-Satellite:-OmniParser) ⦁︎ [Open Design](https://github.com/av/harbor/wiki/2.3.91-Satellite-Open-Design) [Open Interpreter](https://github.com/av/harbor/wiki/2.3.7-Satellite:-Open-Interpreter) ⦁︎ [Open Notebook](https://github.com/av/harbor/wiki/2.3.59-Satellite-Open-Notebook) ⦁︎ [Open Terminal](https://github.com/av/harbor/wiki/2.3.75-Satellite-Open-Terminal) ⦁︎ [Open WebUI Pipelines](https://github.com/av/harbor/wiki/2.3.25-Satellite:-Open-WebUI-Pipelines) [OpenClaw](https://github.com/av/harbor/wiki/2.3.70-Satellite-OpenClaw) ⦁︎ [OpenCode](https://github.com/av/harbor/wiki/2.3.68-Satellite-OpenCode) ⦁︎ [OpenFang](https://github.com/av/harbor/wiki/2.3.73-Satellite-OpenFang) ⦁︎ [OpenHands](https://github.com/av/harbor/wiki/2.3.20-Satellite:-OpenHands) [OptiLLM](https://github.com/av/harbor/wiki/2.3.33-Satellite:-OptiLLM) ⦁︎ [Perplexica](https://github.com/av/harbor/wiki/2.3.2-Satellite:-Perplexica) ⦁︎ [PhotoPrism](https://github.com/av/harbor/wiki/2.3.66-Satellite-PhotoPrism) ⦁︎ [Plandex](https://github.com/av/harbor/wiki/2.3.4-Satellite:-Plandex) [Postiz](https://github.com/av/harbor/wiki/2.3.72-Satellite-Postiz) ⦁︎ [Presenton](https://github.com/av/harbor/wiki/2.1.13-Frontend-Presenton) ⦁︎ [Promptfoo](https://github.com/av/harbor/wiki/2.3.28-Satellite:-Promptfoo) ⦁︎ [Qdrant](https://github.com/av/harbor/wiki/2.3.26-Satellite:-Qdrant) [RAGLite](https://github.com/av/harbor/wiki/2.3.39-Satellite-RAGLite) ⦁︎ [Repopack](https://github.com/av/harbor/wiki/2.3.22-Satellite:-Repopack) ⦁︎ [Resume Matcher](https://github.com/av/harbor/wiki/2.3.55-Satellite-Resume-Matcher) ⦁︎ [ROS MCP Server](https://github.com/av/harbor/wiki/2.3.78-Satellite-ROS-MCP-Server) [SearXNG](https://github.com/av/harbor/wiki/2.3.1-Satellite:-SearXNG) ⦁︎ [Sim Studio](https://github.com/av/harbor/wiki/2.3.58-Satellite-Sim-Studio) ⦁︎ [Solo CLI](https://github.com/av/harbor/wiki/2.3.77-Satellite-Solo-CLI) ⦁︎ [SQL Chat](https://github.com/av/harbor/wiki/2.3.35-Satellite-SQL-Chat) [SuperGateway](https://github.com/av/harbor/wiki/2.3.44-Satellite-supergateway) ⦁︎ [SurfSense](https://github.com/av/harbor/wiki/2.3.85-Satellite-SurfSense) ⦁︎ [TextGrad](https://github.com/av/harbor/wiki/2.3.12-Satellite:-TextGrad) ⦁︎ [tokscale](https://github.com/av/harbor/wiki/2.3.86-Satellite-Tokscale) [Traefik](https://github.com/av/harbor/wiki/2.3.37-Satellite-traefik) ⦁︎ [txtai RAG](https://github.com/av/harbor/wiki/2.3.11-Satellite:-txtai-RAG) ⦁︎ [Unsloth](https://github.com/av/harbor/wiki/2.3.51-Satellite-Unsloth) ⦁︎ [Unsloth Studio](https://github.com/av/harbor/wiki/2.3.83-Satellite-Unsloth-Studio) [Webtop](https://github.com/av/harbor/wiki/2.3.29-Satellite:-Webtop) ⦁︎ [Windmill](https://github.com/av/harbor/wiki/2.3.52-Satellite-Windmill) See [services documentation](https://github.com/av/harbor/wiki/2.-Services) for a brief overview of each. ## CLI Tour ```bash # Run Harbor with default services: # Open WebUI and Ollama harbor up # Run Harbor with additional services # Running SearXNG automatically enables Web RAG in Open WebUI harbor up searxng # Speaches includes OpenAI-compatible SST and TTS # and connected to Open WebUI out of the box harbor up speaches # Run additional/alternative LLM Inference backends # Open Webui is automatically connected to them. harbor up llamacpp tgi litellm vllm tabbyapi aphrodite sglang ktransformers # Host-native backends for macOS (Metal acceleration, no containers) harbor up dmr harbor up mlx harbor up omlx # Run different Frontends harbor up librechat chatui bionicgpt hollama # Get a free quality boost with # built-in optimizing proxy harbor up boost # Use FLUX in Open WebUI in one command harbor up comfyui # Use custom models for supported backends harbor llamacpp model https://huggingface.co/user/repo/model.gguf # Manage models for host-native backends harbor dmr pull ai/mistral harbor dmr ls harbor mlx pull mlx-community/Qwen3.5-4B-4bit harbor omlx pull mlx-community/Qwen3.5-4B-4bit # Access service CLIs without installing them # Caches are shared between services where possible harbor hf scan-cache harbor hf download google/gemma-2-2b-it harbor ollama list # Shortcut to HF Hub to find the models harbor hf find gguf gemma-2 # Use HFDownloader and official HF CLI to download models harbor hf dl -m google/gemma-2-2b-it -c 10 -s ./hf harbor hf download google/gemma-2-2b-it # Where possible, cache is shared between the services harbor tgi model google/gemma-2-2b-it harbor vllm model google/gemma-2-2b-it harbor aphrodite model google/gemma-2-2b-it harbor tabbyapi model google/gemma-2-2b-it-exl2 harbor mistralrs model google/gemma-2-2b-it harbor opint model google/gemma-2-2b-it harbor sglang model google/gemma-2-2b-it # Convenience tools for docker setup harbor logs llamacpp harbor exec llamacpp ./scripts/llama-bench --help harbor shell vllm # Tell your shell exactly what you think about it harbor opint harbor aider harbor aichat harbor cmdh # Use fabric to LLM-ify your linux pipes cat ./file.md | harbor fabric --pattern extract_extraordinary_claims | grep "LK99" # Open services from the CLI harbor open webui harbor open llamacpp # Print yourself a QR to quickly open the # service on your phone harbor qr # Feeling adventurous? Expose your Harbor # to the internet harbor tunnel # Config management harbor config list harbor config set webui.host.port 8080 # Create and manage config profiles harbor profile save l370b harbor profile use default # Import profile from a URL harbor profile use https://example.com/path/to/harbor-profile.env # Lookup recently used harbor commands harbor history # Eject from Harbor into a standalone Docker Compose setup # Will export related services and variables into a standalone file. harbor eject searxng llamacpp > docker-compose.harbor.yml # Run a built-in LLM benchmark with # your own tasks harbor bench run # Gimmick/Fun Area # Argument scrambling, below commands are all the same as above # Harbor doesn't care if it's "vllm model" or "model vllm", it'll # figure it out. harbor model vllm harbor vllm model harbor config get webui.name harbor get config webui_name harbor tabbyapi shell harbor shell tabbyapi # 50% gimmick, 50% useful # Ask harbor about itself harbor how to ping ollama container from the webui? ``` ## Harbor App Demo https://github.com/user-attachments/assets/a5cd2ef1-3208-400a-8866-7abd85808503 In the demo, Harbor App is used to launch a default stack with [Ollama](./2.2.1-Backend:-Ollama) and [Open WebUI](./2.1.1-Frontend:-Open-WebUI) services. Later, [SearXNG](./2.3.1-Satellite:-SearXNG) is also started, and WebUI can connect to it for the Web RAG right out of the box. After that, [Harbor Boost](./5.2.-Harbor-Boost) is also started and connected to the WebUI automatically to induce more creative outputs. As a final step, Harbor config is adjusted in the App for the [`klmbr`](./5.2.-Harbor-Boost#klmbr---boost-llm-creativity) module in the [Harbor Boost](./5.2.-Harbor-Boost), which makes the output unparsable for the LLM (yet still undetstandable for humans). ## Why? - If you're comfortable with Docker and Linux administration - you likely don't need Harbor to manage your local LLM environment. However, while growing it - you're also likely to eventually arrive to a similar solution. I know this for a fact, since that's exactly how Harbor came to be. - Harbor is not designed as a deployment solution, but rather as a helper for the local LLM development environment. It's a good starting point for experimenting with LLMs and related services. - Workflow/setup centralisation - you can be sure where to find a specific config or service, logs, data and configuration files. - Convenience factor - single CLI with a lot of services and features, accessible from anywhere on your host. ## Supporters     <a href="https://x.com/TheAhmadOsman"><img src="docs/supporters-ahmad-osman.png" width="32" height="32" alt="@TheAhmadOsman" /></a>