local-ai-stack：一条命令在本地跑起私有 AI 全家桶

Run your own private AI stack locally with one command.

Quickstart • Features • Presets • CLI Commands • API Gateway • Examples • Documentation

Overview

local-ai-stack is a developer-friendly, zero-friction CLI and container orchestration toolkit that boots a complete, private AI environment directly on your local workstation.

Powered by Docker Compose v2, local-ai-stack gives you local LLMs, chat UI, vector search, database persistence, and an OpenAI-compatible API gateway in seconds. Prompts, chats, and embeddings are processed on your machine; pulling container images and model weights requires an internet connection.

npx local-ai-stack init
npx local-ai-stack up

Features

• Local Processing: Prompts, chats, embeddings, and inference run on your hardware. Initial container image pulls and model downloads require internet access.
• ⚡ Zero Setup Friction: Automatically detects OS, architecture, RAM, and GPU (Apple Metal / NVIDIA CUDA).
• Modular Presets: Scale from a lightweight chat interface (minimal) to an enterprise RAG stack (standard) or an autonomous automation pipeline (full).
• OpenAI-Compatible Gateway: Point standard OpenAI SDKs or LangChain directly to http://localhost:8000/v1 with zero code rewrites.
• Built-in Model Manager: Pull, list, inspect, and remove models with built-in RAM/disk safety warnings.
• Self-Healing Diagnostics: Built-in doctor command diagnoses port collisions, Docker daemon status, and resource limits before you run into issues.

Architecture

┌───────────────────────────────────────────────────────────────┐
│                      Host Machine (CLI)                       │
│                                                               │
│   Port 3000                 Port 8000           Port 5678     │
│       │                         │                   │         │
└───────┼─────────────────────────┼───────────────────┼─────────┘
        ▼                         ▼                   ▼
  ┌───────────┐             ┌───────────┐       ┌───────────┐
  │ Open      │             │ API       │       │ n8n       │
  │ WebUI     │             │ Gateway   │       │ (Full)    │
  └─────┬─────┘             └─────┬─────┘       └─────┬─────┘
        │                         │                   │
        ├─────────────────────────┼───────────────────┤
        ▼                         ▼                   ▼
  ┌───────────┐             ┌───────────┐       ┌───────────┐
  │ Ollama    │             │ Qdrant    │       │ Postgres  │
  │ (LLMs)    │             │ (Vector)  │       │ (State)   │
  └───────────┘             └───────────┘       └───────────┘

Presets

Quickstart

Automated One-Command Installation

macOS & Linux

curl -fsSL https://raw.githubusercontent.com/rasidi3112/local-ai-stack/main/scripts/install.sh | bash

Windows (PowerShell)

iwr -useb https://raw.githubusercontent.com/rasidi3112/local-ai-stack/main/scripts/install.ps1 | iex

Manual Quickstart via npx

Prerequisites: Node.js 22 or later, Docker, and Docker Compose v2.

1. Initialize the stack

# Interactive setup wizard with hardware auto-detection
npx local-ai-stack init

# Or non-interactive with a chosen preset
npx local-ai-stack init --preset standard --yes

2. Start services with automated health checks

npx local-ai-stack up

Once running, access your stack:

• Open WebUI: http://localhost:3000
• API Gateway: http://localhost:8000
• Ollama: http://localhost:11434
• Qdrant: http://localhost:6333/dashboard (when mapped)

3. Smart Hardware Detection & Model Recommendations

# Get smart hardware-aware recommendations based on your RAM/VRAM
npx local-ai-stack model recommend

# Install model with pre-flight memory and disk safety check
npx local-ai-stack model install llama3.2:3b

CLI Commands

Images are pinned to explicit versions. local-ai-stack update refreshes the configured versions; it does not select newer releases. After installing a newer CLI, run local-ai-stack update --upgrade-images to migrate only the CLI's legacy default image tags. Custom image values are preserved.

API Gateway

The API Gateway provides drop-in OpenAI REST endpoints:

Set LOCAL_AI_API_KEY to the generated API_SECRET_KEY before calling a /v1/* endpoint.

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LOCAL_AI_API_KEY" \
  -d '{
    "model": "llama3.2:3b",
    "messages": [
      { "role": "user", "content": "Hello from my private local AI stack!" }
    ]
  }'

Using with the Official OpenAI Node.js SDK

import OpenAI from 'openai';

const apiKey = process.env.LOCAL_AI_API_KEY;
if (!apiKey) {
  throw new Error('Set LOCAL_AI_API_KEY to your generated API_SECRET_KEY.');
}

const openai = new OpenAI({
  baseURL: 'http://localhost:8000/v1',
  apiKey,
});

const completion = await openai.chat.completions.create({
  model: 'llama3.2:3b',
  messages: [{ role: 'user', content: 'Explain quantum computing in one sentence.' }],
});

console.log(completion.choices[0].message.content);

Examples

• examples/openai-compat/: Drop-in OpenAI SDK integration script.
• examples/rag-example/: End-to-end RAG pipeline using Qdrant vector database and Ollama embeddings.

Documentation

• Architecture Overview
• API Gateway Reference
• Configuration Guide
• GPU Acceleration Setup (Metal / CUDA)
• Troubleshooting & Diagnostics

Contributing

Contributions are welcomed! Check out CONTRIBUTING.md to get started.

License

MIT © 2026 local-ai-stack contributors