About airunner

Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows

c

Published by

README.md

AI Runner

Edge AI inference engine with a web GUI — LLMs, image generation, voice chat, and agents running entirely on your hardware, at the edge.

🐞 Report Bug · ✨ Request Feature · 🛡️ Report Vulnerability · 📖 Wiki

What is AI Runner?

AI Runner is a privacy-first edge AI platform — all inference runs locally on your own hardware, not in the cloud. It runs a Python backend that handles model inference and exposes a REST API, paired with a React web frontend you access in your browser. Your prompts, images, and voice data never leave your machine.

Architecture at a glance:

client/                ← React + Vite frontend (port 5173)
server/src/            ← Python inference backend (port 8080)

✨ Features

Feature	Description
🤖 LLM Chat	Local LLMs via llama.cpp (GGUF), with optional OpenRouter/OpenAI backends
🗣️ Voice Chat	Real-time speech-to-text and text-to-speech for hands-free conversations
🎨 Image Generation	Stable Diffusion (SD 1.5, SDXL) and FLUX with LoRA and inpainting
🧠 AI Agents	Configurable personalities, moods, RAG-enhanced memory, and tool use
🔒 Privacy First	Runs fully offline by default — no data leaves your machine
🌐 Web UI	React frontend, accessible from any browser on your local network
⚡ Optimized	GGUF quantization, attention slicing, and VRAM offloading for lower-end hardware

⚙️ System Requirements

	Minimum	Recommended
OS	Ubuntu 22.04	Ubuntu 24.04
CPU	Ryzen 2700K / i7-8700K	Ryzen 5800X / i7-11700K
RAM	16 GB	32 GB
GPU	NVIDIA RTX 3060	NVIDIA RTX 4080+
Storage	22 GB SSD	100 GB+ SSD
Python	3.13.3+	3.13.3+

🚀 Quick Start

Install

Clone the repo and run the install script:

git clone https://github.com/Capsize-Games/airunner.git
cd airunner
./scripts/install.sh

This installs the Python backend and all frontend dependencies.

Run

./scripts/run_web.sh

Then open your browser at http://localhost:5173.

The backend API is available at http://localhost:8080.

Logs

All server and runtime (art, TTS, STT, LLM) logs go to a single file:

tail -f build/logs/server.log

💾 Manual Installation (Advanced)

If you need fine-grained control, the install script supports three modes:

# Developer mode — installs from source (default for contributors)
./scripts/install.sh

# Distributed mode — for server/multi-machine deployments
./deployment/install_distributed.sh

# Single-package mode — installs a prebuilt self-contained bundle
./package/build_bundle.sh

Python dependencies

Python 3.13.3+ is required. We recommend pyenv + venv.

Install PyTorch first:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

Then install the backend package:

pip install -e "server/src/.[core,llm-native,stt-native,art-python,tts-python]"

llama-cpp-python (CUDA build)

CMAKE_ARGS="-DGGML_CUDA=on -DGGML_CUDA_ARCHITECTURES=90" FORCE_CMAKE=1 \
  pip install --no-binary=:all: --no-cache-dir "llama-cpp-python==0.3.21"

90 targets RTX 4090/5080-class GPUs. Drop -DGGML_CUDA_ARCHITECTURES to auto-detect your GPU.

🤖 Models

Essential TTS/STT models download automatically on first run. LLM and image models must be configured manually.

Category	Model	Size
LLM (default)	Ministral-8B-Instruct (GGUF)	~4 GB
Image	Stable Diffusion 1.5	~2 GB
Image	SDXL 1.0	~6 GB
Image	FLUX.1 Dev/Schnell (GGUF)	8–12 GB
TTS	OpenVoice	654 MB
STT	Whisper Tiny	155 MB

Place art models in ~/.local/share/airunner/art/models/.

🔒 HTTPS

The local server uses HTTPS by default. Certificates are auto-generated at ~/.local/share/airunner/certs/.

For browser-trusted certificates, install mkcert:

sudo apt install libnss3-tools
mkcert -install
airunner-generate-cert

🧪 Testing

# Run the full test suite
airunner-tests

# Run daemon-safe tests directly
pytest server/src/

# With coverage
airunner-test-coverage-report

⚖️ Colorado AI Act Notice

Effective February 1, 2026, the Colorado AI Act (SB 24-205) regulates high-risk AI systems. If you use AI Runner to make decisions with legal or significant effects on individuals (employment screening, loan eligibility, housing, etc.), you may be classified as a deployer of a high-risk AI system and subject to compliance obligations.

AI Runner is designed to run fully locally with no external data transmission by default. Optional features that do connect externally: model downloads (HuggingFace/CivitAI), web search (DuckDuckGo), weather prompts (Open-Meteo), and external LLM providers (OpenRouter/OpenAI) if configured. We recommend using a VPN when using these features.

airunner