llm-parliament：让三个 AI 模型按议会流程辩论你的问题

Multi-agent debate for better AI decisions. Research-backed, local-first.

Three AI models debate your question through a parliamentary process: First Reading, Debate, and Division. The result is a structured verdict with consensus, split views, risks, and a recommendation.

Built on multi-agent debate, a technique shown to improve AI accuracy by 7-15% in research (Liang et al. 2023, Chen et al. 2023). Our own live tests, including where it does not help, are summarised under When it helps — measured.

See CHANGELOG.md for the release history.

👋 This is my GitHub and developer debut! llm-parliament is the first project I've shipped publicly. I built it to learn, and I'd genuinely love your help making it better.

Feedback, bug reports, feature ideas, code review, and pull requests are very welcome — no contribution is too small. If something feels confusing, doesn't install, breaks on your terminal, or could be more polished, please open an issue or say hi in Discussions.

Concrete places to start: good first issues · help wanted · roadmap · CONTRIBUTING.md

Stars also help me see what's resonating. Thanks for taking a look. 🙏

Quick Start

pipx install llm-parliament
parliament doctor
parliament              # opens the TUI

The mock parliament runs out of the box with no setup.

Installation

The recommended way is pipx — it installs the tool into an isolated environment but exposes parliament globally on your PATH, so you don't have to think about virtual environments.

Linux

# Prereqs (one-time)
sudo apt install pipx          # Debian/Ubuntu — or `pacman -S python-pipx`, `dnf install pipx`
pipx ensurepath                # adds ~/.local/bin to PATH
# Restart your shell.

# Install
pipx install llm-parliament

# Verify
parliament doctor

macOS

# Prereqs (one-time)
brew install pipx
pipx ensurepath
# Restart your shell.

# Install
pipx install llm-parliament

# Verify
parliament doctor

Windows

# Prereqs (one-time)
# 1. Install Python 3.11+ from python.org — check "Add Python to PATH" during install.
# 2. Install pipx:
python -m pip install --user pipx
python -m pipx ensurepath
# 3. Close and reopen Windows Terminal (recommended) or PowerShell.

# Install
pipx install llm-parliament

# Verify
parliament doctor

Notes:

• All cloud provider SDKs (Anthropic, Google, OpenAI) are bundled. No extras needed.
• Ollama (for local models) is a separate native daemon — install from https://ollama.com if you want local models. The parliament doctor command tells you what's detected.
• On Windows, the install pulls in windows-curses automatically so the TUI works out of the box. Windows Terminal is recommended over cmd.exe (better VT/UTF-8 support); legacy cmd.exe is supported.
• On Windows, redirecting parliament doctor or parliament ask output to a file (> out.txt) currently crashes with UnicodeEncodeError (#74). Until it's fixed, run $env:PYTHONIOENCODING = "utf-8" first. parliament ask --json is not affected.
• Keys are stored in the OS native credential store (Windows Credential Manager, macOS Keychain, GNOME Keyring) via parliament keys set. Falls back to ~/.parliament/keys.env if no keyring is available.

Verify your install

After install, run:

parliament doctor

You'll see something like:

Environment
  ✓ Python 3.12.5 (>=3.11 required)
  ✓ Curses available
  ✓ Terminal: 142x38
  ✓ Config: ~/.parliament/config.yaml (initialized)

Providers
  ✓ Anthropic SDK         ℹ ANTHROPIC_API_KEY not set
  ✓ Google SDK            ℹ GOOGLE_API_KEY not set
  ✓ OpenAI SDK            ℹ OPENAI_API_KEY not set
  ℹ Ollama: not reachable at http://localhost:11434

Next steps
  - Add cloud keys:   parliament keys set <provider> <key>
  - Local models?     Install Ollama from https://ollama.com, then `ollama pull llama3.1`
  - Run the TUI:      parliament

Exit code is 0 if the install is functional (regardless of whether you have keys/Ollama configured), or 1 if something is broken (e.g. Python too old).

Configuration

Your personal config lives at ~/.parliament/config.yaml (or %USERPROFILE%\.parliament\config.yaml on Windows) — outside the repo, so your settings never end up in git.

On first run (triggered by parliament doctor, parliament ask, or launching the TUI), a setup wizard runs automatically. It detects your API keys, local Ollama models, and available RAM, then proposes a matching preset:

Welcome to Parliament. No config found - let's set one up.

Detected:
  ✓ ANTHROPIC_API_KEY (configured)
  ℹ OPENAI_API_KEY (not set)
  ✓ GOOGLE_API_KEY (configured)
  ℹ OPENROUTER_API_KEY (not set)
  ℹ Ollama: not reachable
  System RAM: 16 GB

Proposed preset: cloud-anthropic-google (Anthropic + Google)
  - Claude (anthropic / claude-sonnet-4-6)
  - Claude-Haiku (anthropic / claude-haiku-4-5-20251001)
  - Gemini (google / gemini-2.5-flash)

Use these defaults? [Y/n]:

Confirm with Y and the config is written. If no keys or Ollama are detected, you get a working mock preset — no setup needed to verify the install. Edit members later via the TUI (parliament) or directly in the file.

With OPENROUTER_API_KEY, the wizard can propose three labs through one account. Two or three direct provider keys keep their existing presets, and three reachable local models that fit RAM take priority over OpenRouter. Otherwise OpenRouter wins over mixed, single-provider and mock presets. Its proposal warns that API usage may incur charges. Declining any proposal writes a mock preset, not the next available cloud or local preset.

docs/configuration.md is the complete reference: every key, its type, its real default, and the environment variable or CLI flag that overrides it — plus settings.json, which the TUI writes and this file does not cover.

Choosing the Speaker

The Speaker writes the final verdict. By default it is one of the members: the strongest by tier, or the first listed when they tie. In our tests a member Speaker tended to side with its own position, even with member names hidden from it. A Speaker that did not debate has no position to defend:

parliament:
  members:
    - {name: Claude, provider: openrouter, model: anthropic/claude-sonnet-5.5}
    - {name: GPT,    provider: openrouter, model: openai/gpt-5.6-sol}
    - {name: Gemini, provider: openrouter, model: google/gemini-3.1-pro-preview}
  speaker:
    name: Grok
    provider: openrouter
    model: x-ai/grok-4.7

Pick a model that is not on the panel. parliament ask warns when the Speaker shares a member's model, and when tied members leave the choice to list order. If the outside Speaker fails, a member takes over, and --speaker <member> still overrides it.

Optional: Local Models (Ollama)

Ollama runs LLMs locally — free, private, no API keys. Install it separately, then point a parliament member at it.

1. Install Ollama from https://ollama.com and start the daemon.
2. Pull a model:

   ollama pull llama3.1

3. Edit ~/.parliament/config.yaml (or use the TUI) to add an Ollama member:

   parliament:
     members:
       - name: Llama
         provider: ollama
         model: llama3.1
   providers:
     ollama:
       base_url: http://localhost:11434/v1

4. Run parliament to start a debate.

Note: All providers default to no timeout (timeout: null), so a slow local model on modest hardware won't be cut off. If you want a hard limit, set timeout: 600.0 on the relevant providers.<name> block.

Optional: Cloud Models

Easiest path — export your keys in your shell profile before the first run and the wizard picks them up automatically:

export ANTHROPIC_API_KEY=sk-ant-...
export GOOGLE_API_KEY=...
parliament doctor   # wizard fires, detects keys, writes a cloud preset

After first run — add or change keys at any time:

parliament keys set anthropic sk-ant-...
parliament keys set google ...
parliament keys set openai sk-...

Keys are saved to the OS native credential store. Then edit members via the TUI (parliament) or directly in ~/.parliament/config.yaml:

parliament:
  members:
    - name: Claude
      provider: anthropic
      model: claude-sonnet-4-6
    - name: Gemini
      provider: google
      model: gemini-2.5-flash

You can also export ANTHROPIC_API_KEY, OPENAI_API_KEY, and GOOGLE_API_KEY directly in your environment instead of using the keyring.

Useful key commands:

parliament keys list
parliament keys migrate   # move existing keys.env entries to the OS keyring
parliament keys remove openai

Optional: OpenAI-compatible providers (Groq, Mistral, OpenRouter)

Groq and Mistral speak the OpenAI API and are available directly as groq and mistral providers. Store each key separately:

parliament keys set groq gsk_...
parliament keys set mistral your-mistral-key

Both providers can coexist with OpenAI and OpenRouter in the same parliament:

parliament:
  members:
    - name: Groq
      provider: groq
      model: llama-3.3-70b-versatile
    - name: Mistral
      provider: mistral
      model: mistral-large-latest

They appear in the TUI provider picker and /key. parliament doctor checks the required keys for configured Groq and Mistral members; they do not have separate rows in its Providers section. Groq reads GROQ_API_KEY; Mistral reads MISTRAL_API_KEY. A missing vendor key never falls back to OPENAI_API_KEY, including during model discovery. Optional providers.groq and providers.mistral entries can override the base_url or supply an explicit api_key.

Existing provider: openai configurations with a custom base_url still work, using providers.openai.api_key or OPENAI_API_KEY as before. Move to the named providers to keep vendor credentials and endpoints separate.

Their models retain their tier labels and the usual tier-gap warnings.

OpenRouter

OpenRouter is also OpenAI-compatible, but it is common enough to deserve its own line in parliament keys, its own button in the TUI provider picker, and its own row in parliament doctor. Name it directly:

parliament keys set openrouter sk-or-...

parliament:
  members:
    - name: Claude
      provider: openrouter
      model: anthropic/claude-sonnet-4.6

The address (https://openrouter.ai/api/v1) and the key variable (OPENROUTER_API_KEY) come from model_catalog.OPENAI_COMPATIBLE; you do not set a base_url. The TUI's model picker fetches OpenRouter's live catalogue the same way it does for Anthropic, OpenAI, and Google.

Other OpenAI-compatible endpoints

Any service speaking the OpenAI API that is not its own first-class provider can use provider: openai: point providers.openai at its base_url and give it a key. Aggregators such as one-api, self-hosted gateways, and small vendors that ship only an OpenAI-shaped endpoint all fall into this category.

When it helps — measured

We ran live tests on Windows through OpenRouter (October 2026). The full method, every run and the raw tallies are in docs/live-test-windows.md. In short:

• Easy or settled questions: no gain. When the answer is a best practice (how to store passwords, say), all three members agree, and one good model gives the same answer faster and cheaper.
• An uneven panel acts like its strongest member. With one strong model and two cheap ones, the cheap members came round to the strong one. The verdict was the strong model's answer, at about 3× the time.
• Contested judgement calls: a consistent way to decide, not a fixed answer. On a question where experts genuinely differ, with three evenly matched frontier models and an outside Speaker, 8 debates chose one option 6 times and another twice. All 8 gave the same first step and the same conditions for switching. In 5 of 8 a member changed its mind on a peer's argument. A single model asked the same question gave different answers depending on the wording.
• The Speaker matters. Re-judging the same 8 debates, three outside Speakers agreed on 7. A Speaker running the same model as a member sided with that member. See Choosing the Speaker.

Use it for decisions with real trade-offs, and read the Split and the conditions for switching, not only the recommended option.

Does it cost 3× more?

Yes — Parliament makes more API calls than asking a single model: 3 for First Reading, 3 for Debate, 1 for Division (7 total). On cloud APIs, that's real money.

Approximate cost per query:

Measured on OpenRouter in October 2026: under $0.10 a debate for Sonnet 4.6, GPT-4o-mini and Gemini 2.5 Flash, and about $0.25 for Sonnet 5.5, GPT-5.6 Sol and Gemini 3.1 Pro with an outside Speaker. A debate took about 1.5–3 minutes; a single call to one of those models took 20–50 seconds.

The right question is whether that cost is worth it for the specific decision.

Parliament is designed for decisions where being wrong is expensive — architecture choices, technical trade-offs, strategy calls. Getting those wrong can cost days or weeks of rework. The token cost of a debate is a rounding error compared to the cost of a wrong call.

For quick lookups, summaries, or anything with an obvious answer, use a single model. Parliament is a deliberation tool, not a throughput tool.

Three reasons the cost argument flips:

1. Tier mixing cuts the cost, but also the disagreement. One strong model plus two cheap ones (as in the OpenRouter wizard preset) costs much less than three strong ones. In our tests, though, the cheap members came round to the strong one, so the verdict was mostly that model's answer. For a real debate, use evenly matched members.

2. Local models make it free. Ollama runs 3B–13B models on commodity hardware at no API cost. For anyone who can run two small local models, the "3×" concern disappears entirely.

3. One good answer beats three mediocre ones. A single-model answer on a hard architectural question has blind spots the model doesn't know it has. The debate surfaces them. If it prevents one wrong architectural decision, it has paid for months of usage.

Three ways to keep costs low:

The first-run wizard (parliament doctor on a fresh install) automatically suggests a cost-aware preset based on what keys and local models you have available. You can also mix cloud and local: e.g. one Anthropic member + two Ollama members keeps the cloud bill minimal while still getting the debate benefit.

Rule of thumb: if the cost of being wrong on this decision exceeds $1, the debate is worth it.

CLI Usage

# Check that the install is healthy
parliament doctor

# Use mock providers for fast local testing
parliament ask "Is this architecture too complex?" --mock

# Show the full transcript before the verdict (post-hoc dump)
parliament ask "Which queue should we use?" --verbose

# Print the full Hansard as JSON for scripts and jq pipelines
parliament ask "Which queue should we use?" --json

# Hide the live debate panels and only print the final verdict
parliament ask "Quick check?" --no-show-debate

# Make a member the Speaker (overrides parliament.speaker; see "Choosing the Speaker")
parliament ask "What are the main risks?" --speaker Claude

# Show configured members
parliament members

# Open the full TUI dashboard
parliament

# Open the TUI with mock providers, no Ollama/API keys needed
parliament --mock

# Browse the same dashboard with a specific config
parliament tui --config /path/to/custom-config.yaml

Live debate view

By default, parliament ask and the curses TUI render the debate live: a panel pops in for each member as their analysis lands, and stage headers mark the transitions through First Reading → Debate → Division. This makes it obvious which model is currently working and what they said.

The view is toggleable via three precedence-ordered sources:

--show-debate controls whether the live panels appear during the run.

Hansard detail levels

By default the post-run terminal output is verdict — the full four-part synthesis: what the members agreed on, where they split, the risks any of them flagged, and the Speaker's recommendation. The split is the part a single model cannot give you, so it ships without anyone having to ask for it. Prefer the recommendation alone? --hansard=minimal. Saved .md files are always written at archive level regardless of the display level, so nothing is lost from the record either way. Older runs that printed the full debate text are opt-in via --hansard=full.

Four levels:

Set the level via three precedence-ordered sources:

--verbose continues to work; it's an alias for --hansard=full. The level applies to the post-run terminal output only — saved .md files are always written at archive, and --json is never gated by level. The live in-flight debate view is independent — toggle it separately with --show-debate / --no-show-debate.

For machine-readable output, parliament ask --json prints the complete Hansard object. See docs/hansard-schema.md for the JSON fields and practical jq examples.

TUI controls:

Type                Edit the question field
Enter               Run debate when question is focused
Tab                 Switch between question and members
Up/down or j/k      Move through models
Enter               Open selected member settings when members are focused
s                   Open settings when members are focused; save result from verdict screen
e                   Edit the selected member from the detail view
Left/backspace/Esc  Return from settings to dashboard
Ctrl+U              Clear the question field
Ctrl+S              Save member edits
q                   Quit when focused on members/settings
Ctrl+Q              Quit from anywhere

The TUI settings screen lets you set the local directory used for saved Hansard Markdown responses. By default, saved responses go to ~/.parliament/hansards.

Member editing stays inside the TUI. The editor lets you change Name, Provider, Model, and Base URL, while Tier, Role, and API key status remain derived and read-only. Model pickers include supported presets plus a Custom model escape hatch.

Inside the member editor, Enter opens provider/model pickers when those fields are focused and saves the edit when the base URL field is focused.

Development

Clone the repo and create a virtual environment.

Linux / macOS:

git clone https://github.com/elarmuzik1993/llm-parliament.git
cd llm-parliament

python -m venv .venv
source .venv/bin/activate
python -m pip install -U pip
python -m pip install -e ".[dev]"

Windows (PowerShell):

git clone https://github.com/elarmuzik1993/llm-parliament.git
cd llm-parliament

python -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install -U pip
python -m pip install -e ".[dev]"

(In cmd.exe, use .venv\Scripts\activate.bat instead.)

Run the test suite:

python -m pytest

Run Ruff:

ruff check .

Try the CLI without external services:

parliament ask "What should we test first?" --mock

Project Layout

src/parliament/
  cli.py                  Click CLI (parliament ask / doctor / keys / members / tui / update)
  commands.py             Slash commands (/help, /update, /doctor, /history, /copy, …)
  config.py               YAML config loading, key management, resolve_* precedence helpers
  doctor.py               Health-check logic (Python, curses, terminal, providers, Ollama)
  model_catalog.py        Provider model presets + tier data for pickers
  tui.py                  Curses TUI — all screens, key handling, main loop
  core/                   Parliament orchestrator, dataclasses, ProgressEvent
  procedures/             First Reading, Debate, and Division phases
  providers/              Adapters for Ollama, OpenAI, Anthropic, Google, plus Mock
  render/                 HansardLevel + terminal/markdown renderers + live CLI/TUI views
tests/                    Pytest suite (pytest-asyncio)
docs/
  configuration.md        Every config key, default and override
  hansard-schema.md       JSON schema emitted by `parliament ask --json`
  live-test-windows.md    Live test plan and results (what the debate adds, and when)
scripts/
  diagnose-render.py      Render diagnostic — colors + spinner debugging
config.example.yaml       Default config template (fallback if first-run wizard fails)
config.cloud.yaml         Cloud-only example (Anthropic + Google + OpenAI)
config.mixed.yaml         Mixed example (Ollama + cloud)
AGENTS.md                 Contributor source-of-truth (architecture, conventions)
CHANGELOG.md              Release history (Keep-a-Changelog)
RELEASING.md              PyPI release procedure

Contributing

Yes, please. This project is early and there is a lot of easy ground to cover — bug reports, docs fixes, terminal-compatibility reports, and code are all welcome, and small PRs are genuinely fine.

The whole test suite runs on mock providers, so you can develop and test without an API key or a running Ollama:

python -m pip install -e ".[dev]"
python -m pytest -q
parliament ask "What should we test first?" --mock

CI runs pytest and ruff on Linux, macOS, and Windows across Python 3.11–3.13, so a PR gets checked on platforms you may not have.

For AI agents and automated tools

This repo is meant to be navigable without reading every file:

• AGENTS.md — the machine-readable source of truth: repository layout, the debate pipeline, threading model, config precedence, conventions, and step-by-step recipes for adding a provider or a slash command. Agents read it directly; there is no per-tool copy.
• docs/hansard-schema.md — the JSON schema emitted by parliament ask --json, with jq recipes.
• --mock — every entry point accepts it, giving deterministic output with no network access, which makes the project safe to run in a sandbox.
• Project Layout below and the table in CONTRIBUTING.md map change areas to the files that own them.

Disclaimer

LLM Parliament is an orchestration framework. It coordinates multiple AI models to provide structured debate and synthesis. Users are responsible for complying with the Terms of Service and Usage Policies of their respective LLM providers (e.g., Ollama, OpenAI, Anthropic, Google). This tool does not bypass safety filters or usage restrictions of the underlying models.

Privacy

LLM Parliament is a local tool with no telemetry, analytics, or remote reporting of any kind. The author receives no data from your usage.

What leaves your machine and where it goes:

What never leaves your machine:

• API keys — stored in the OS keyring or ~/.parliament/keys.env, never logged or included in any output
• Saved Hansard files — written to ~/.parliament/hansards/ locally only
• Config — ~/.parliament/config.yaml is read locally, never transmitted

You can verify this by inspecting the source: all outbound calls are in src/parliament/providers/ (debate traffic to your providers) and src/parliament/model_catalog.py (model list fetches to your providers). There are no other network calls in the codebase.

License

AGPLv3