跳到正文
GitHub·· 7 天前AI 评分36

archestack/best-of-ai:155 个开源 AI 项目排名榜

archestack/best-of-ai

AI 导读

archestack 发布 best-of-ai 开源 AI 项目排名榜,收录 155 个项目、分 16 个类别,数据截至 2026-10-10,每天重新排名两次。

正文

Archestack

Open-source AI, ranked. 🏆

Apps, agents, models and the tools around them: what each one does, what it needs, where it breaks. No fluff, and paid never moves a rank.

projects stars categories updated

Best of Open-Source AI is a ranked list of 155 open-source AI projects: chat apps and assistants, desktop apps, agent platforms and frameworks, workflow automation, RAG, memory, model serving, open models, training, image, video and voice, vector databases, gateways, observability, sandboxes, data tools, MCP servers and learning material. Each entry has strengths, weaknesses, specs and a score from public GitHub signals. Updated 2026-10-10 (UTC).

Tip

🔎 Every project fact-checked on GitHub before it is listed  ·  ✍️ Every entry has strengths and weaknesses  ·  ▶️ Demo links so you can try before you deploy

Note

🤝 Independent. We sell none of the tools listed here, and nobody can pay to move up. Ranks come from public GitHub data and are re-checked twice a day; hover a score to see what it is made of.

Looking for coding agents, starters and the stack you ship on? 👉 Best of Vibe Coding

🔥 Top 10 right now

155 projects ranked in 16 categories · data from 2026-10-10 · re-ranked twice a day

How to read it: rank 🥇🥈🥉 then #⁠4, #⁠5… (equal scores: more stars first) · score out of 100 sets the order: hover it to see what it is made of, or read how we rank · click a project name for its review · ↗ opens another site (Cmd/Ctrl-click for a new tab)

Rank Project Score Category Stars
🥇 OmniRoute
OpenAI-compatible gateway that routes requests across hundreds of AI providers
🐳 Docker · MIT · 🌐 Site ↗
86 🔀 Gateways 75k
🥈 LiteLLM
OpenAI-format gateway and Python SDK for calling 100+ LLM providers
🐳 Docker · custom license · 📖 Docs ↗ · 🌐 Site ↗
84 🔀 Gateways 61k
🥉 nanobot
Small Python agent runtime with bundled WebUI, TUI and chat channels
🐳 Docker · MIT · 📖 Docs ↗
84 🤖 Assistants 49k
#⁠4 LocalAI
One OpenAI-compatible server for text, speech, image and video models
🐳 Docker · MIT · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
82 🧠 Model serving 49k
#⁠5 Milvus
Distributed vector database with dense, sparse and hybrid search at scale
🐳 Docker · Apache-2.0 · GPU optional · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
81 🧮 Vector databases 46k
#⁠6 LobeHub
Agent workspace with builder, groups, scheduling and 10,000+ MCP skills
🐳 Docker · custom license
80 💬 Chat UIs 83k
#⁠7 OpenClaw
Personal assistant gateway that answers in Discord, Slack, WhatsApp and Telegram
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
79 🤖 Assistants 392k
#⁠8 llama.cpp
C/C++ inference engine serving GGUF models over an OpenAI-compatible API
🐳 Docker · MIT · GPU optional · 🌐 Site ↗
79 🧠 Model serving 131k
#⁠9 hindsight
Agent memory server with retain, recall and reflect operations
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
78 🗂️ Memory 48k
#⁠10 Lightpanda
Headless browser written in Zig for AI agents and scraping
🐳 Docker · AGPL-3.0 · 📖 Docs ↗ · 🌐 Site ↗
78 🛡️ Sandboxes 36k

Ordered by score: adoption, freshness, how easy it is to run, and a few more signals as we measure them (how we rank).

⚡ Pick by vibe

🔓 Open-source alternatives to …

The top open-source picks by score in the matching categories. ✋ means the project's own README calls it an alternative to that product.

Instead of Try these open-source picks
ChatGPT LobeHub · AnythingLLM · Open WebUI
Perplexity Morphic · Local Deep Research · GPT Researcher
LM Studio LocalAI · llama.cpp · vLLM
Glean LightRAG · Open Notebook · RAGFlow
Copilot Studio Langflow · Dify · AutoGPT
NotebookLM Open Notebook ✋ · SurfSense ✋ · LightRAG
Zapier n8n · Activepieces
OpenAI API LocalAI · llama.cpp · vLLM
OpenRouter OmniRoute · LiteLLM · freellmapi
LangSmith Langfuse · Phoenix · promptfoo
Pinecone Milvus · Meilisearch · Chroma
Midjourney ComfyUI · MoneyPrinterTurbo · InvokeAI
ElevenLabs Voicebox ✋ · Speech-to-Speech · Pocket TTS
Otter.ai Voicebox · Speech-to-Speech · Pocket TTS
Mem hindsight · MemPalace · agentmemory

💻 Runs on what you have

From each project README: whether it needs a GPU, and whether it mentions Apple Silicon or arm64. Apps and tools you run, not libraries.

What you have Projects Top picks
No GPU needed 102 OmniRoute · LiteLLM · nanobot · LobeHub · OpenClaw · Lightpanda
A GPU helps, not required 33 LocalAI · Milvus · llama.cpp · Open WebUI · ComfyUI · vLLM
Needs a GPU 8 KTransformers · GPUStack · AI Toolkit · LMDeploy · Kohya's GUI · TabbyAPI
A Mac or another arm64 machine 57 OmniRoute · nanobot · LocalAI · Milvus · llama.cpp · hindsight

🗂️ All categories

🧑‍💻 Apps you run

🧱 Build with it

🧑‍💻 Apps you run

🤖 Assistants

Personal AI assistants you run yourself and talk to through chat apps, with memory and the ability to act. 12 ranked · 📝 all reviews

nanobot leads with 84, ahead of OpenClaw (79) and Hermes Agent (76).

Rank Project Score Stars
🥇 nanobot
Small Python agent runtime with bundled WebUI, TUI and chat channels
🐳 Docker · MIT · 📖 Docs ↗
84 49k
🥈 OpenClaw
Personal assistant gateway that answers in Discord, Slack, WhatsApp and Telegram
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
79 392k
🥉 Hermes Agent
Terminal and chat-app agent that writes its own skills and remembers you
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
76 252k
#⁠4 ZeroClaw
Single Rust binary agent runtime with 30+ channels and hardware access
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
75 33k
#⁠5 cowagent
Personal AI agent with memory, skills and chat channels like Telegram and Slack
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
66 47k
#⁠6 QwenPaw
Personal AI agent with memory, skills, and chat-channel connectors
🐳 Docker · Apache-2.0 · GPU optional · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
66 36k
#⁠7 AstrBot
Chatbot platform bridging LLMs to QQ, Telegram, Discord, Slack and more
🐳 Docker · AGPL-3.0 · 📖 Docs ↗
65 42k
#⁠8 IronClaw
Rust assistant that sandboxes every untrusted tool in WebAssembly
🐳 Docker · Apache-2.0
65 13k
#⁠9 Moltis
Persistent personal agent server in one Rust binary with sandboxed execution
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
59 2.9k
#⁠10 Khoj
Personal assistant that chats with your documents and the web
🐳 Docker · AGPL-3.0 · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
56 38k

Top 10 of 12. See all 12, ranked, with reviews →

💡 How to choose

  • Check which channels it supports today (WhatsApp, Telegram, Slack, Discord, email) and whether each needs a paid API.
  • Look at how memory is stored and whether you can inspect or wipe it.
  • Actions need credentials; prefer assistants that scope them per tool.

↑ categories

💬 Chat UIs

Web front-ends for local or API models, usually with user accounts, chat history and file upload. 13 ranked · 📝 all reviews

LobeHub leads with 80, ahead of AnythingLLM (77) and Open WebUI (76).

Rank Project Score Stars
🥇 LobeHub
Agent workspace with builder, groups, scheduling and 10,000+ MCP skills
🐳 Docker · custom license
80 83k
🥈 AnythingLLM
Document chat and agent app with built-in RAG, MCP and multi-user support
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
77 67k
🥉 Open WebUI
Self-hosted chat UI for Ollama and OpenAI-compatible APIs with RBAC and RAG
🐳 Docker · custom license · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
76 154k
#⁠4 big-AGI
Multi-model chat workspace with Beam side-by-side model comparison
🐳 Docker · MIT · 🌐 Site ↗
71 7.1k
#⁠5 Hermes WebUI
Browser UI for Hermes Agent with sessions and file browser
🐳 Docker · MIT · 🌐 Site ↗
69 19k
#⁠6 LibreChat
Multi-provider ChatGPT-style app with agents, MCP, code interpreter and auth
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
66 45k
#⁠7 Onyx
Team knowledge chat that indexes 50+ apps for RAG and agents
🐳 Docker · custom license · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
66 32k
#⁠8 NextChat
Lightweight Next.js chat client for OpenAI, Claude, Gemini and DeepSeek APIs
🐳 Docker · MIT · ▶️ Demo ↗ · 🌐 Site ↗
64 89k
#⁠9 SillyTavern
Local chat front end for role-play across many LLM backends
🐳 Docker · AGPL-3.0 · 📖 Docs ↗
58 34k
#⁠10 HuggingChat UI
SvelteKit chat front end behind HuggingChat for OpenAI-compatible endpoints
🐳 Docker · Apache-2.0 · ▶️ Demo ↗
57 11k

Top 10 of 13. See all 13, ranked, with reviews →

💡 How to choose

  • Confirm it talks to your backend (Ollama, OpenAI-compatible, Anthropic) without a plugin.
  • Multi-user auth, RBAC and SSO are where free and paid editions differ most.
  • Check what the default compose pulls in (database, vector store) before sizing the host.

↑ categories

🧩 Agent platforms

Visual or code-first builders for agents and workflows, with orchestration, tools and deployment. 10 ranked · 📝 all reviews

Langflow leads with 73, ahead of Dify (72) and AutoGPT (70).

Rank Project Score Stars
🥇 Langflow
Visual flow builder that deploys agents as APIs or MCP servers
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
73 155k
🥈 Dify
Visual LLM app platform with workflows, RAG pipeline, agents and APIs
🐳 Docker · custom license · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
72 158k
🥉 AutoGPT
Block-based builder for agents that run on demand, schedule or trigger
custom license · ▶️ Demo ↗ · 📖 Docs ↗
70 188k
#⁠4 Sim
Workspace to build, deploy and monitor agents with 1,000+ integrations
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
70 30k
#⁠5 Multica
Issue board where AI coding agents take assignments like teammates
🐳 Docker · custom license · 📖 Docs ↗ · 🌐 Site ↗
68 52k
#⁠6 Paperclip
Task manager and org chart for teams of AI agents with budgets
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
67 100k
#⁠7 Skyvern
Browser automation agent driven by vision LLMs over Playwright
🐳 Docker · AGPL-3.0 · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
66 23k
#⁠8 Agent Zero
Agent framework that gives the model a full Linux desktop in Docker
🐳 Docker · custom license · 🌐 Site ↗
62 19k
#⁠9 FastGPT
Knowledge-base Q&A and visual workflow platform for LLM apps
🐳 Docker · custom license · 📖 Docs ↗ · 🌐 Site ↗
61 30k
#⁠10 Botpress
SDK, CLI and open-source integrations for the Botpress Cloud bot platform
🐳 Docker · MIT · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
38 15k

💡 How to choose

  • Decide between a visual builder (faster to start) and code-first (easier to test and version).
  • Check how workflows are exported; vendor-specific JSON makes migration costly.
  • Look at the license for the server part; several are AGPL or source-available.

↑ categories

🔁 Workflow automation with AI

Workflow and integration platforms with AI steps or agent nodes, for wiring models into the rest of your tools. 2 ranked · 📝 all reviews

n8n leads with 76, ahead of Activepieces (65).

Rank Project Score Stars
🥇 n8n
Visual workflow automation with code steps, AI agent nodes and 1500+ integrations
🐳 Docker · custom license · 📖 Docs ↗
76 207k
🥈 Activepieces
Self-hosted workflow automation with TypeScript integrations, alternative to Zapier
🐳 Docker · custom license · 📖 Docs ↗ · 🌐 Site ↗
65 25k

💡 How to choose

  • Count the integrations you actually need; the long tail is where platforms differ.
  • Check whether AI steps can use your own model endpoint or only hosted providers.
  • Read the license: several are fair-code or source-available, not OSI open source.

↑ categories

📚 RAG and knowledge

Document Q&A, knowledge bases and enterprise search over your own files and data. 15 ranked · 📝 all reviews

LightRAG leads with 76, ahead of Open Notebook (75) and RAGFlow (72).

Rank Project Score Stars
🥇 LightRAG
Graph-plus-vector RAG server with web UI and Ollama-compatible API
🐳 Docker · MIT
76 40k
🥈 Open Notebook
Self-hosted NotebookLM alternative with podcasts and 20+ model providers
🐳 Docker · MIT · 🌐 Site ↗
75 40k
🥉 RAGFlow
RAG engine with document parsing, agentic retrieval and citations
🐳 Docker · Apache-2.0 · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
72 92k
#⁠4 WeKnora
Enterprise knowledge base combining RAG Q&A, agents and generated wikis
🐳 Docker · custom license · 📖 Docs ↗ · 🌐 Site ↗
68 33k
#⁠5 SurfSense
Offline NotebookLM alternative that turns documents into decks, reports and podcasts
🐳 Docker · custom license · 📖 Docs ↗ · 🌐 Site ↗
68 16k
#⁠6 MaxKB
Enterprise knowledge-base agent platform with RAG, workflows and MCP tools
🐳 Docker · GPL-3.0
66 23k
#⁠7 PrivateGPT
Anthropic-style API layer for private RAG on local inference servers
🐳 Docker · Apache-2.0 · 📖 Docs ↗
63 58k
#⁠8 PipesHub
Permission-aware search and agent context over 50+ workplace systems
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
57 3.8k
#⁠9 DeepWiki-Open
Generates browsable wikis and diagrams for GitHub, GitLab and Bitbucket repos
🐳 Docker · MIT · 🌐 Site ↗
55 18k
#⁠10 DB-GPT
Agentic data assistant that writes SQL and code over your databases
🐳 Docker · MIT · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
54 20k

Top 10 of 15. See all 15, ranked, with reviews →

💡 How to choose

  • Match the ingestion formats you need (PDF with tables, Office, web, Confluence) before anything else.
  • Check which embedding models and vector stores are supported and whether they can run offline.
  • Look for citations in answers; without them, RAG output is hard to trust.

↑ categories

🔎 Search

Private search engines and AI answer engines that keep queries on your host. 5 ranked · 📝 all reviews

Morphic leads with 70, ahead of Local Deep Research (65) and GPT Researcher (64).

Rank Project Score Stars
🥇 Morphic
Search engine that answers with citations and renders rich inline components
🐳 Docker · Apache-2.0
70 9.2k
🥈 Local Deep Research
Agentic research assistant with local LLMs, SearXNG and encrypted libraries
🐳 Docker · MIT · GPU optional
65 9.2k
🥉 GPT Researcher
Research agent that writes cited reports from web and local documents
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
64 30k
#⁠4 Vane
Self-hosted answer engine with cited sources over SearXNG
🐳 Docker · MIT
55 37k
#⁠5 MAESTRO
Multi-agent research platform that writes long reports from documents and web
🐳 Docker · AGPL-3.0 · GPU optional · 📖 Docs ↗
30 1.5k

💡 How to choose

  • Metasearch engines need upstream providers; check rate limits and whether results are cached.
  • Answer engines call an LLM per query; budget tokens before exposing them to a team.
  • Check how results are attributed; answer engines without citations are hard to verify.

↑ categories

🎙️ Voice

Speech-to-text, text-to-speech, voice agents and meeting tools that run locally. 11 ranked · 📝 all reviews

Voicebox leads with 72, ahead of Speech-to-Speech (67) and Pocket TTS (64).

Rank Project Score Stars
🥇 Voicebox
Local voice studio for cloning, TTS, dictation and agent speech
🐳 Docker · MIT · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
72 57k
🥈 Speech-to-Speech
Modular voice-agent pipeline exposed through the OpenAI Realtime API
🐳 Docker · Apache-2.0 · GPU optional · 📖 Docs ↗
67 13k
🥉 Pocket TTS
100M-parameter CPU text-to-speech with streaming and voice cloning
🐳 Docker · MIT · ▶️ Demo ↗ · 📖 Docs ↗
64 9.9k
#⁠4 Kokoro-FastAPI
OpenAI-compatible Kokoro-82M speech API in CPU and GPU images
🐳 Docker · Apache-2.0 · GPU optional · ▶️ Demo ↗
64 5.5k
#⁠5 IndexTTS
Zero-shot TTS with emotion, speed and pronunciation control
custom license · ▶️ Demo ↗
61 24k
#⁠6 GPT-SoVITS
Few-shot voice cloning and TTS with a training web UI
🐳 Docker · MIT · GPU optional · ▶️ Demo ↗ · 📖 Docs ↗
59 63k
#⁠7 F5-TTS
Flow-matching TTS and voice cloning with Gradio and CLI
🐳 Docker · MIT · ▶️ Demo ↗
58 15k
#⁠8 OpenReader
Reads EPUB, PDF and DOCX aloud with synced word highlighting
🐳 Docker · MIT · 📖 Docs ↗
56 540
#⁠9 Speakr
Transcribe, summarize and search recordings with pluggable ASR and LLMs
🐳 Docker · AGPL-3.0 · 📖 Docs ↗
52 4.1k
#⁠10 WhisperLive
Near-real-time Whisper transcription server over WebSocket
🐳 Docker · MIT · GPU optional
49 4.3k

Top 10 of 11. See all 11, ranked, with reviews →

💡 How to choose

  • Latency decides usability for voice agents; check the end-to-end numbers the project publishes.
  • Most quality models need a GPU; CPU-only setups are slower and limited to smaller models.
  • Check language coverage and whether models are downloaded at build or at first run.

↑ categories

🎨 Image and video

Generation UIs and pipelines for images and video, usually around diffusion models. 8 ranked · 📝 all reviews

ComfyUI leads with 75, ahead of MoneyPrinterTurbo (69) and InvokeAI (62).

Rank Project Score Stars
🥇 ComfyUI
Node-graph engine for diffusion image, video, audio and 3D models
GPL-3.0 · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
75 137k
🥈 MoneyPrinterTurbo
Generates short videos from a topic with script, footage, voice and subtitles
🐳 Docker · MIT
69 129k
🥉 InvokeAI
Canvas-first web UI for Stable Diffusion and Flux image generation
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
62 29k
#⁠4 AI Toolkit
Fine-tuning suite for image, video and audio diffusion models, with GUI and CLI
🐳 Docker · MIT · GPU required
54 12k
#⁠5 Kohya's GUI
Gradio GUI and CLI for Kohya diffusion training scripts
🐳 Docker · Apache-2.0 · GPU required
52 13k
#⁠6 Pixelle-Video
Topic-to-short-video pipeline built on ComfyUI workflows and TTS
🐳 Docker · Apache-2.0 · 📖 Docs ↗
51 29k
#⁠7 biniou
Chat, image, audio, video and 3D generation in one CPU-friendly web UI
🐳 Docker · GPL-3.0 · GPU optional · 📖 Docs ↗
49 1.2k
#⁠8 FluxGym
Web UI for training FLUX LoRAs on 12 to 20 GB GPUs
🐳 Docker · MIT · GPU required
39 3.3k

💡 How to choose

  • A GPU with enough VRAM is the hard requirement; check the minimum the project states.
  • Node-based UIs are flexible but have a learning curve; form-based UIs are quicker to use.
  • Check the model licensing separately from the app licensing.

↑ categories

🔐 Security

AI agents and tools for penetration testing, red-teaming and finding vulnerabilities in your own apps and models. 1 ranked · 📝 all reviews

Strix leads with 69.

Rank Project Score Stars
🥇 Strix
Autonomous AI pentesting agents that validate findings with working exploits
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
69 68k

💡 How to choose

  • Run them only against targets you own or are authorized to test; check the tool's scope controls.
  • Check which model backends are supported and whether findings stay on your host.
  • Look at how exploitation is sandboxed (containers, VMs) before pointing it at production.

↑ categories

🧱 Build with it

🗂️ Memory

Long-term memory engines that store and retrieve facts for agents across sessions. 12 ranked · 📝 all reviews

hindsight leads with 78, ahead of MemPalace (77) and agentmemory (75).

Rank Project Score Stars
🥇 hindsight
Agent memory server with retain, recall and reflect operations
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
78 48k
🥈 MemPalace
Local verbatim memory for coding agents on ChromaDB with 45 MCP tools
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
77 59k
🥉 agentmemory
Persistent memory server for coding agents, exposed over MCP and REST
🐳 Docker · Apache-2.0
75 29k
#⁠4 OpenViking
Context database exposing agent memory, knowledge and skills as a filesystem
🐳 Docker · AGPL-3.0 · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
74 40k
#⁠5 Mem0
Memory layer for agents with a self-hosted server, SDKs and CLI
Apache-2.0 · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
73 67k
#⁠6 Cognee
Memory engine that turns documents and code into a knowledge graph
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
73 32k
#⁠7 Graphiti
Temporal knowledge graph framework for agent memory with REST and MCP servers
🐳 Docker · Apache-2.0
70 32k
#⁠8 MemOS
Memory operating system for agents with cubes, scheduler and hybrid retrieval
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
65 12k
#⁠9 Supermemory
Memory and context API with user profiles, connectors and a local server
MIT · 📖 Docs ↗
63 31k
#⁠10 Honcho
Memory service modelling users, agents and groups as evolving peers
🐳 Docker · AGPL-3.0 · ▶️ Demo ↗ · 📖 Docs ↗
56 7.6k

Top 10 of 12. See all 12, ranked, with reviews →

💡 How to choose

  • Check what gets stored (raw messages, extracted facts, graphs) and how it is retrieved.
  • Multi-tenant isolation matters if several users or agents share one store.
  • Look at the backing store (Postgres, a vector DB, a graph DB) and whether you already run it.

↑ categories

🧠 Model serving

Inference engines and model servers that expose local models over an API. 20 ranked · 📝 all reviews

LocalAI leads with 82, ahead of llama.cpp (79) and vLLM (75).

Rank Project Score Stars
🥇 LocalAI
One OpenAI-compatible server for text, speech, image and video models
🐳 Docker · MIT · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
82 49k
🥈 llama.cpp
C/C++ inference engine serving GGUF models over an OpenAI-compatible API
🐳 Docker · MIT · GPU optional · 🌐 Site ↗
79 131k
🥉 vLLM
High-throughput LLM serving engine with OpenAI and Anthropic APIs
🐳 Docker · Apache-2.0 · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
75 94k
#⁠4 Ollama
Runs open-weight models locally behind a CLI and REST API
🐳 Docker · MIT · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
74 183k
#⁠5 colibri
C inference engine that runs huge MoE models by streaming experts from disk
🐳 Docker · Apache-2.0 · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
70 41k
#⁠6 SGLang
Inference server for LLM, vision-language and diffusion models
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
68 37k
#⁠7 Lemonade
Local AI server that targets GPUs and AMD NPUs with OpenAI-style APIs
🐳 Docker · Apache-2.0 · GPU optional
66 5.9k
#⁠8 exo
Distributed LLM inference across Macs and other devices, MLX-based
Apache-2.0 · GPU optional · 📖 Docs ↗
64 48k
#⁠9 llama-swap
Go proxy that hot-swaps local model servers per request
🐳 Docker · MIT · GPU optional
63 5.9k
#⁠10 Xinference
Serves LLM, embedding, speech and image models behind one OpenAI-compatible API
🐳 Docker · Apache-2.0 · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
62 9.6k

Top 10 of 20. See all 20, ranked, with reviews →

💡 How to choose

  • Pick by hardware first (CPU, NVIDIA, AMD, Apple Silicon) and by model format (GGUF, safetensors).
  • Throughput engines (continuous batching, paged attention) need GPUs; single-user servers do not.
  • An OpenAI-compatible API keeps the rest of your stack portable.

↑ categories

🔀 Gateways

LLM gateways and proxies for routing, caching, rate limits and cost control across providers. 14 ranked · 📝 all reviews

OmniRoute leads with 86, ahead of LiteLLM (84) and freellmapi (76).

Rank Project Score Stars
🥇 OmniRoute
OpenAI-compatible gateway that routes requests across hundreds of AI providers
🐳 Docker · MIT · 🌐 Site ↗
86 75k
🥈 LiteLLM
OpenAI-format gateway and Python SDK for calling 100+ LLM providers
🐳 Docker · custom license · 📖 Docs ↗ · 🌐 Site ↗
84 61k
🥉 freellmapi
OpenAI-compatible router that fails over across free LLM provider tiers
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
76 33k
#⁠4 ContextForge MCP Gateway
Registry and proxy federating MCP, A2A, REST and gRPC behind one endpoint
🐳 Docker · Apache-2.0 · 📖 Docs ↗
68 4.6k
#⁠5 Higress
Envoy-based API gateway with LLM proxy plugins and MCP server hosting
🐳 Docker · Apache-2.0 · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
61 9.5k
#⁠6 Bifrost
Go AI gateway with web UI, fallbacks, budgets and semantic caching
🐳 Docker · Apache-2.0 · 📖 Docs ↗
59 8.7k
#⁠7 Plano
Envoy-based data plane that routes, traces and guards agent traffic
🐳 Docker · Apache-2.0 · 📖 Docs ↗
58 7.1k
#⁠8 GoModel
Go AI gateway with OpenAI and Anthropic APIs, caching and budgets
🐳 Docker · MIT · ▶️ Demo ↗ · 📖 Docs ↗
57 1.2k
#⁠9 agentgateway
One proxy for LLM, MCP and A2A traffic with auth and RBAC
🐳 Docker · Apache-2.0 · 📖 Docs ↗
54 5.3k
#⁠10 optillm
OpenAI-compatible proxy applying inference-time reasoning techniques
🐳 Docker · Apache-2.0 · GPU optional · ▶️ Demo ↗
51 4.3k

Top 10 of 14. See all 14, ranked, with reviews →

💡 How to choose

  • List the providers you call today; check the gateway's native support rather than generic passthrough.
  • Key management, per-team budgets and audit logs separate gateways from simple proxies.
  • Check latency overhead and whether streaming is passed through unchanged.

↑ categories

📈 Observability

Tracing, evaluation and prompt management for LLM applications. 13 ranked · 📝 all reviews

Langfuse leads with 74, ahead of Phoenix (74) and promptfoo (71).

Rank Project Score Stars
🥇 Langfuse
Tracing, prompt management and evals for LLM apps on ClickHouse
🐳 Docker · custom license · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
74 36k
🥈 Phoenix
LLM tracing, evals, datasets and prompt playground built on OpenTelemetry
🐳 Docker · custom license · 📖 Docs ↗ · 🌐 Site ↗
74 12k
🥉 promptfoo
CLI for evaluating and red-teaming prompts, agents and RAG
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
71 26k
#⁠4 MLflow
Tracing, evals, prompt registry and AI gateway plus classic ML tracking
🐳 Docker · Apache-2.0 · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
70 28k
#⁠5 Opik
Trace, evaluate and monitor LLM apps and agents, Apache-2.0 end to end
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
64 22k
#⁠6 Latitude
Agent observability that groups failures and dispatches coding agents to fix them
🐳 Docker · MIT · 📖 Docs ↗ · 🌐 Site ↗
62 4.7k
#⁠7 Future AGI
Tracing, evals, simulation, guardrails and an LLM gateway for AI agents
🐳 Docker · Apache-2.0 · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
61 2.1k
#⁠8 LangWatch
Agent observability, simulation testing, AI gateway and governance in one
Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
60 4.9k
#⁠9 Laminar
Rust-based agent tracing with SQL queries, signals and evals
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
57 3.4k
#⁠10 OpenLIT
OpenTelemetry-based tracing, evals and guardrails for LLM apps and coding agents
🐳 Docker · Apache-2.0 · GPU optional · 📖 Docs ↗ · 🌐 Site ↗
57 2.8k

Top 10 of 13. See all 13, ranked, with reviews →

💡 How to choose

  • Confirm SDK support for your framework (OpenTelemetry, LangChain, OpenAI SDK) and your language.
  • Evaluation features vary widely; check whether evals run on your data without a cloud account.
  • Retention and storage backend decide the host size; traces grow fast.

↑ categories

🧮 Vector databases

Vector stores and hybrid search engines for embeddings. 8 ranked · 📝 all reviews

Milvus leads with 81, ahead of Meilisearch (69) and Chroma (64).

Rank Project Score Stars
🥇 Milvus
Distributed vector database with dense, sparse and hybrid search at scale
🐳 Docker · Apache-2.0 · GPU optional · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
81 46k
🥈 Meilisearch
Rust search engine API with full-text, vector and hybrid search
🐳 Docker · custom license · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
69 60k
🥉 Chroma
Embedding database with a four-function API for Python and JavaScript
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
64 29k
#⁠4 Qdrant
Rust vector database with payload filtering, REST and gRPC
🐳 Docker · Apache-2.0 · GPU optional · ▶️ Demo ↗ · 📖 Docs ↗
63 35k
#⁠5 pgvector
PostgreSQL extension for vector similarity search with HNSW and IVFFlat
🐳 Docker · custom license
59 23k
#⁠6 Weaviate
Go vector database with built-in vectorizers, hybrid search and RAG
🐳 Docker · custom license · ▶️ Demo ↗ · 📖 Docs ↗
57 17k
#⁠7 HelixDB
Rust graph database with native vector and BM25 search
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
55 6.2k
#⁠8 Vespa
Serving engine for vectors, tensors, text and ML ranking at scale
Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
41 7.1k

💡 How to choose

  • If you already run Postgres, check pgvector-based options before adding a new database.
  • Hybrid search (keyword + vector) and filtering are where engines differ most.
  • Check memory use per million vectors; it decides the host size.

↑ categories

🛡️ Sandboxes

Isolated runtimes where agents execute code, browse or use tools safely. 8 ranked · 📝 all reviews

Lightpanda leads with 78, ahead of Obscura (71) and NemoClaw (71).

Rank Project Score Stars
🥇 Lightpanda
Headless browser written in Zig for AI agents and scraping
🐳 Docker · AGPL-3.0 · 📖 Docs ↗ · 🌐 Site ↗
78 36k
🥈 Obscura
Rust headless browser with CDP, native rendering and stealth mode
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
71 29k
🥉 NemoClaw
NVIDIA reference stack running OpenClaw and Hermes inside OpenShell sandboxes
🐳 Docker · Apache-2.0 · 📖 Docs ↗
71 23k
#⁠4 OpenShell
Policy-enforced sandbox runtime for autonomous agents with credential brokering
🐳 Docker · Apache-2.0 · 📖 Docs ↗
69 16k
#⁠5 microsandbox
Local microVMs for untrusted code with fork, snapshot and SDKs
🐳 Docker · Apache-2.0 · 📖 Docs ↗
61 8.6k
#⁠6 Steel Browser
Browser API that manages Chrome sessions for Puppeteer, Playwright and Selenium
🐳 Docker · Apache-2.0 · 📖 Docs ↗ · 🌐 Site ↗
60 7.8k
#⁠7 Browser Use Web UI
Gradio UI for running browser-use agents with your own Chrome
🐳 Docker · MIT · 📖 Docs ↗
48 17k
#⁠8 Open Terminal
REST-driven shell and file sandbox for AI agents, from Open WebUI
🐳 Docker · MIT
46 3.3k

💡 How to choose

  • Isolation level (container, microVM, gVisor) sets the risk you accept when agents run code.
  • Check startup time per sandbox; slow starts limit agent loops.
  • Look at what is persisted between runs and how secrets are injected.

↑ categories

🕸️ Data and scraping for AI

Tools that turn web pages, PDFs and documents into clean text or structured data that models can use. 3 ranked · 📝 all reviews

Crawl4AI leads with 72, ahead of Firecrawl (68) and Jina Reader (44).

Rank Project Score Stars
🥇 Crawl4AI
Python crawler that turns pages into LLM-ready markdown, with a Docker API
🐳 Docker · Apache-2.0 · 📖 Docs ↗
72 85k
🥈 Firecrawl
Web scraping and crawling API that returns LLM-ready markdown
🐳 Docker · AGPL-3.0 · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
68 190k
🥉 Jina Reader
Converts any URL or search query into LLM-friendly markdown
🐳 Docker · Apache-2.0 · ▶️ Demo ↗ · 📖 Docs ↗ · 🌐 Site ↗
44 12k

💡 How to choose

  • Test on your hardest pages (JavaScript-heavy, tables, scans) before choosing.
  • Check whether it runs fully self-hosted or needs the vendor's API for parts.
  • Look at output formats: markdown, JSON with a schema, or both.

↑ categories

🏅 How we rank

Every project gets a score out of 100 from the signals below. A signal we cannot measure yet is left out and its weight goes to the others, so nobody loses points for things we have not checked. The score sets the order; 🥇🥈🥉 mark the top three places in each table.

Signal Weight What it looks at Shown as (80+ · 50+ · 30+ · lower) Status
⭐ Adoption 15% GitHub stars, log-scaled, ranked half within the category and half across the list widely used · popular · known · niche live
📈 Momentum 15% stars gained in the last 30 days, ranked within the category surging · rising · steady · flat soon
🕒 Freshness 15% last commit (full marks under 90 days, zero at a year) and last release active · recent · slowing · quiet live
🔧 Maintenance 10% issues closed vs opened, release cadence, commits in 90 days healthy · fair · patchy · weak live
🚀 Easy to run 20% apps and tools: a way to install it (Docker, pip, npm, binary, script, Helm or a release download); starters: GitHub template; both: one-click deploy, arm64, healthcheck. Not scored for libraries, plugins, models and collections very easy · easy · some setup · hard live
✅ Verified build 10% we start it in CI and wait for a healthy response passes · untested · fails soon
📖 Docs 5% install section, environment docs, upgrade notes thorough · good · thin · missing soon
🤖 Agent-ready 5% CLAUDE.md, AGENTS.md, skills, llms.txt ready · partly · minimal · none live
🔒 Privacy 5% no telemetry or opt-out, works fully offline private · mostly private · some telemetry · leaky soon

Penalties: no license −10 · data must leave your host −10 · star-growth anomaly −15 · unpatched security advisory −20. Archived projects are removed. Paid placements never touch the score.

The weights are our judgment, not science. Think one is wrong? Tell us. Method changes: 2026-10-09: Docker alone earns no points, medals mark places 1–3 instead of score bands.

🧠 How this works

  • 🧠 On this list: open-source projects whose main job is AI, from apps you run to libraries you build with. Tools for coding with AI, starters and the stack you ship on live on the sister list, Best of Vibe Coding.
  • 🔎 Found in curated lists, app stores and template galleries, then fact-checked on GitHub: stars, last commit, license, Docker files.
  • ✍️ Written from the README, never copied from other lists: what it does, what it needs, strengths and weaknesses as claims you can check. Specs say unknown rather than guess.
  • 🔄 Kept fresh: entries are rewritten when the README or the latest release changes; projects quiet for 12 months get marked stale, archived ones drop off.
  • 🚫 No pay-to-rank. Sponsors and affiliate links, if any, are labeled and never touch the order.

📬 Your turn

➕ Add a project · 🛠️ Fix wrong data · 🚪 Opt out (done within 24 hours). Everything here is generated, so use the forms instead of editing files.

📚 Where candidates come from

These lists, app stores and galleries (facts and links only, no text copied), plus community submissions: awesome-selfhosted · awesome-local-llm · awesome-homelab · self-hosting-guide · awesome-ai-agent-platforms · awesome-openclaw · umbrel-apps · runtipi-appstore · casaos-appstore · dokploy-templates · gh-search-self-hosted-ai · gh-search-ai-agents · gh-search-llm-apps · gh-search-ai-security · awesome-opensource-ai · awesome-llm-services · awesome-llmops · awesome-private-ai · awesome-local-llms · awesome-llm-webapps · 0xsojalsec-llms-local · inftyai-awesome-llmops · bradagi-awesome-cli-coding-agents · best-of-ml-python · awesome-generative-ai · awesome-ai-agents-e2b · awesome-mcp-servers · gh-search-llm-top · gh-search-ml-top · gh-search-agents-top · gh-search-mcp-top.

📄 License

Data (data/, this README, reviews/) is CC BY 4.0; see LICENSE-DATA. Code is MIT; see LICENSE. Project names and descriptions belong to their owners.

Curated with care, powered by public GitHub data, funded by freelance gigs. Found a gem? Tips keep the servers happy, but rankings stay honest. 🚀
For freelance collabs or coffee-fueled conversations, say hello on LinkedIn.

Made with ☕ by Archestack · see also Best of Vibe Coding

来源:GitHub · github.com