Riko 开源 AI 语音伴侣：50K 快照发布，本地运行带 3D 虚拟形象

Talk to her out loud. She talks back.
An anime AI companion that runs on your own PC, with her own voice and a 3D avatar that lip-syncs every word.

Important

This is the 50K snapshot: a complete Riko, open source and yours to keep. It's frozen at this release, so it stays a stable base to build on.

Riko keeps growing on Patreon, home of the Full Source and the Desktop App. See how they compare.

GitHub issues aren't monitored, so there's no support here.

🦊 Meet Riko

Riko is a voice companion you set up yourself. Press Enter and talk. She hears you, comes up with a reply (usually a snarky one), and says it out loud while her lips move along on screen.

This repo is the open-source core of Project Riko: the whole loop from your voice to hers, written to be read. It's under 500 lines in total, and every part sits in its own small file, so when you want to change how she thinks, sounds or looks, you know exactly where to go.

Her personality, her avatar and her voice all live in one config file. Making her yours takes a few minutes.

✨ What she can do

⚙️ How it works

flowchart LR
    you(["🎙️ You talk"]) --> ears["👂 Ears<br/>Faster-Whisper or Groq"]
    ears --> brain["🧠 Brain<br/>OpenAI GPT"]
    brain --> voice["🗣️ Voice<br/>GPT-SoVITS or Fish Audio"]
    voice --> face(["💃 Face<br/>VRM avatar + lip sync"])

Loading

Each step is one file, and main.py runs them in a loop:

🚀 Quick start

The easiest way in uses Fish Audio for her voice: no GPU and nothing extra to install. If you'd rather generate her voice on your own PC, do this first and then see Local voice.

You need: a Windows or Linux PC with a microphone, an OpenAI API key, and a Fish Audio API key (sign up, then create one under API keys).

1. Install uv. It sets up Python for you.

# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

# Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
sudo apt install libportaudio2    # lets Python use your microphone

Close and reopen your terminal afterwards so it can find uv.

2. Get the code.

git clone https://github.com/rayenfeng/riko_project.git
cd riko_project

No git? Click the green Code button at the top of this page → Download ZIP, unzip it, then open a terminal in that folder. On Windows 11: right-click inside the folder → Open in Terminal.

3. Install everything.

uv sync

4. Add your API keys.

copy .env.example .env    # Linux: cp .env.example .env
notepad .env              # Linux: nano .env

Paste your keys after the = signs and save. .env is git-ignored, so your keys stay out of any commit.

5. Start her.

uv run main.py --tts fish

The first launch takes a minute while it downloads the speech recognition model.

6. Say hi.

1. Open http://127.0.0.1:8080 and click once. Browsers only play sound after a click.
2. Back in the terminal, press Enter, say something, press Enter.
3. She answers on the page. Ctrl+C quits.

Tip

Set tts: fish in character_config.yaml and a plain uv run main.py does the same thing.

🎨 Make her yours

Everything about her lives in character_config.yaml. Put her files (avatar, voice clip) in character_files/.

system_prompt: |
  You are a helpful assistant named Riko.
  You speak like a snarky anime girl.
  Always refer to the user as "senpai".

You can also switch voice or ears for one run without touching the config:

uv run main.py --tts sovits    # or fish
uv run main.py --asr groq      # or local

🎤 Local voice with GPT-SoVITS

Want her voice generated on your own machine, cloned from any clip you like? That's what GPT-SoVITS does. It runs best on an NVIDIA GPU and is installed separately.

1. Install GPT-SoVITS by following its guide.
2. In the GPT-SoVITS folder, start its API server and leave it running:

   python api_v2.py -a 127.0.0.1 -p 9880

3. Start Riko with uv run main.py --tts sovits.

She starts with the sample voice in character_files/main_sample.wav. Swap in your own clip (see the table above) to change it.

💎 Three ways to have Riko

Same Riko, three ways to have her. Each one is made for a different kind of person, so pick the one that matches how you like to work.

Get the Full Source or the Desktop App on Patreon →

📋 Requirements

Tested on Windows 11 and Ubuntu 22.04 (WSL2) with an RTX 3060 12 GB. Faster-Whisper runs on the CPU, so any modern processor handles the ears.

Where the keys come from

🩹 Troubleshooting

🔧 Under the hood

main.py                the loop: record → transcribe → reply → speak → play
riko/
  config.py            loads character_config.yaml and .env
  asr.py               push-to-talk recording + speech to text
  llm.py               OpenAI reply + chat_history.json
  tts.py               GPT-SoVITS or Fish Audio → client/audio/riko.wav
  viewer.py            serves the avatar page and hands it each voice clip
client/
  index.html           three.js + three-vrm: the avatar, lip sync, rotate and zoom
character_files/       her avatar (sample_avatar.vrm) and sample voice (main_sample.wav)
character_config.yaml  her personality, avatar and voice

Every part also runs on its own, which helps when something isn't working:

uv run python -m riko.llm      # one reply from OpenAI
uv run python -m riko.tts      # one voice line (add --fish for Fish Audio)
uv run python -m riko.asr      # record, then print what she heard

Ideas if you want to build on it

• Hands-free listening. Replace record() in riko/asr.py with something that notices when you stop talking.
• Emotions. client/index.html already opens her mouth with vrm.expressionManager.setValue('aa', ...). Expressions like happy and angry work the same way.

🙏 Credits

• Voice by GPT-SoVITS and Fish Audio
• Speech recognition by Faster-Whisper and Groq
• Language model by OpenAI
• Avatar rendering by three.js and three-vrm
• Sample avatar by pixiv Inc. (VRM1_Constraint_Twist_Sample)

📄 License

MIT. Feel free to clone, modify, and build your own waifu voice companion.