Harbor

Harbor is a CLI and companion app that spins up complete pre-wired local LLM stacks with Docker Compose.

Harbor is a CLI and companion application designed to set up and manage local LLM stacks without manual configuration. It allows users to orchestrate backends like Ollama, llama.cpp, and vLLM alongside frontends such as Open WebUI and supporting services like SearXNG, Speaches, and ComfyUI. The tool handles Docker Compose orchestration, connectivity, and configuration automatically. Harbor supports various host coding tools and inference engines with metal acceleration on macOS. It is intended for developers and experimenters who want a centralized local AI development environment.

Harbor at a glance

Best for: Software engineer
Runs on: macOS, CLI, self-hosted, open source
Pricing: Free
Released: July 2024
Category: AI coding tool
Website: github.com/av/harbor

Key features

Spin up local LLM stacks with a single command
Orchestrate Docker Compose services automatically
Chain agentic modules with Harbor Boost workflows
Integrate backends with host coding and agent CLIs
Expose local services to the internet with tunneling
Generate configuration profiles and command history

Use it for

Run local LLMs and chat interfaces
Set up voice chats with local models
Execute local web RAG and deep research
Generate images using ComfyUI and Flux
Script workflows and interactions with downstream LLMs

Alternatives to Harbor

llm-d — Deploys production-grade LLM serving on existing hardware clusters.
Flisk — Eliminates local infrastructure setup pain and AI hallucination.
Manifest LLM Gateway — It autofixes bad requests and routes calls across multiple providers with zero downtime.
HolyClaude — Combines multiple verified AI coding CLIs and browser automation in one Docker workspace.
LLM CLI — It enables command-line interaction with remote APIs and local machine models.
Widefleet — Deploys agent-built apps with built-in company SSO, databases, and file storage.
CLI Manager — Manage multiple AI coding agents and watch visible sessions from a single interface.
LangChain4j — Integrates LLMs and vector stores directly into Java applications.
Ballast — Finds hidden developer clutter across more than 60 tools and safely reclaims disk space.
OpenLLM — Routes developer tools through multiple providers and local models with included inference.
Ferry — Sync your agent configurations across Linux machines without copying tokens or logins.
Portkey — Unified API access with observability, guardrails, and caching for LLM applications.

All Harbor alternatives