Stephen Spielbot 开源 AI 短片生成器发布，从选题到成片全流程自动化

Stephen Spielbot

An AI video generator that turns a topic into a fully produced short film — complete with cinematic visuals, narration or scenes acted out by its characters, and background music.

▶️ Watch how it works

📖 Documentation & manual  ·  ✍️ The story: “I will never direct a movie again”

What it does

1. Script — an LLM (local vLLM, Claude, Grok, or OpenAI) drafts and critiques the whole story as prose, you review it, and it is then divided into scenes with visual prompts, narration, and a mood-matched music description
2. Images — FLUX.2 Klein (the default per-style image engine) generates each scene's first-frame still, with optional recurring characters kept consistent via reference images. Qwen-Image 2.1 is an opt-in alternative for generation, with an NVFP4 build that is the faster one on GB10 workers (research license only; masked edits stay on FLUX)
3. Video — LTX 2.5 animates each scene from its still via ComfyUI (local or distributed workers)
4. Narration — F5-TTS synthesises speech with voice cloning from a reference WAV. The default weights are the Apache-2.0 OpenF5-TTS-Base so narration is licensed for commercial use — see docs/tts_licensing.md. A per-style voice-model picker adds Chatterbox Multilingual (23 languages, with a per-style narration language that also drives the script's language)
5. Dialogue — scenes can instead be acted or silent: the characters speak on screen, with MiniMax H3 Ref2VA generating picture and voice together from their portraits
6. Music — ACE-Step generates background music from the LLM's mood description, mixed in at the very end (switch it off per style or per film). A per-style music-model picker adds MiniMax Music 3 — slower and song-shaped, under its own community license. The Music video format goes further: the LLM writes the story as tagged lyrics, the music model sings them (a lead singer is cast from the style's character catalogue and the vocalist is matched to their sex, age and background), and the cast performs the song on camera — or you upload a song you already have and the film is built around that instead. Re-voice either kind of song with a library voice using SoulX-Singer SVC (default) or Seed-VC, selected per style; both are installed by make install
7. Assembly — FFmpeg mixes everything into a single video with synced audio. Films aimed at QHD/4K (or upscaled later from the film editor) go through FlashVSR video super-resolution by default — see upscaler modes

A film's format decides how the story is staged: narrated throughout, entirely acted — the characters speaking on screen, no narrator and no music — a music video sung end to end, or mixed, with acted, narrated and silent scenes side by side. Each scene then takes the render path its mode asks for — and a style can send its silent scenes down the acted path too, performed on H3 from their own first frame (plus the portraits of anyone in shot) so a wordless beat matches the takes it cuts against.

Around the pipeline, the web app also handles the full channel workflow: a render queue with automation — global or per style, unattended in any format, music videos included, their song written, sung and re-voiced before the pictures are made — AI-suggested video ideas, optional per-style news research in a dedicated News tab using OpenAI, Claude or Grok web tools with existing API keys (or optional X post search), shared cached research and a global daily research cap, with on-demand, AI Ideas page-visit or scheduled checks (default once daily when scheduled), self-contained, sourced directions, people appearance references with best-guess character fallbacks and uncertainty warnings, and unattended news-to-video or news-to-song creation, per-scene editing with image inpainting, clip trimming, per-scene review marks (to be reviewed / to work on / approved) and version history, opening titles and end credits joined onto the finished film, an ending fade that dips the last seconds to black, one-click restyling (the same script or finished film in another visual style), misspelling-proof cover thumbnails (text-free artwork in the style's own engine + real-font typography with per-style fonts, colours, and accent words), publishing to YouTube (multi-channel, with playlists, captions — attached or burned into the picture — and tags) and X, a publish scheduler with per-channel cadence, comment fetching / AI replies / community engagement, a predictive engagement model, and optional per-style video attribution metadata and C2PA "AI-generated" content credentials on published videos.

Quick start

Controller (runs the web app): Python 3.11+, Node.js 20+, FFmpeg, and a local vLLM server or an API key (Claude, Grok, or OpenAI) for script generation. Workers (GPU machines): Docker + the NVIDIA Container Toolkit — ComfyUI and TTS run as containers that make install builds and deploys, so a worker needs no Python of its own. One machine can be both.

git clone https://github.com/pizzato/stephen_spielbot
cd stephen_spielbot
make install WORKERS="s1 s2 s3"   # deps, models, workers, config.yaml, web UI
make start                        # start ComfyUI on all workers, then launch the app

Omit WORKERS=... for a single-machine (localhost) setup, or run make install with no args to be prompted. Then open http://localhost:8001.

make stop      # stop everything
make status    # check health of the app and all workers

Full setup, cluster deployment, and uninstall instructions: Installation.

Security

The web app has no authentication and is meant to run bound to localhost as a single-user tool — anyone who can reach port 8001 has full control. Don't expose it to untrusted networks. make tailscale shares it with your tailnet (no app auth, tailnet-only — never public). See SECURITY.md.

Documentation

Everything lives at pizzato.github.io/stephen_spielbot:

• Installation — requirements, make install, day-to-day commands
• Your first film — topic → script → render → publish, end to end
• The manual — every screen in the web app, control by control
• Configuration · Environment variables · Models
• Cluster & workers · Architecture · Troubleshooting

The source for that site is docs/ — make docs-serve previews it locally. Guides that also read well on GitHub:

• docs/characters.md — recurring characters: consistent looks, reference images, voices
• docs/performance_films.md — acted scenes and performance films: portraits + dialogue straight to video, no first frame or TTS
• docs/orchestration.md — the durable SQLite task layer and how renders execute
• docs/youtube_setup.md — Google Cloud / OAuth setup for YouTube publishing
• docs/x_setup.md — X (Twitter) developer app setup for posting
• docker/README.md — the containerized worker stack in detail
• webapp/README.md — web UI architecture and development workflow
• CONTRIBUTING.md — dev setup, tests, and the CI gate

Channels using this tool

• Stephen Spielbot (@StephenSpielbot) — YouTube · The original
• A Brief History of Botkind (@BHOBk) — YouTube
• A Brief History of Botkind (@aBHOBk) — X
• Amelia and the World (@AmeliaAndTheWorld) — YouTube

Making films with Stephen Spielbot? Add your channel to channels.yaml and open a pull request — that file is the only thing you need to edit. On merge, a GitHub Action regenerates this list and the app's About screen, and opens a follow-up pull request with the result. Run make channels if you want to preview the result locally.

License

Stephen Spielbot's code is licensed under Apache-2.0.

The AI models it downloads each carry their own licenses — see THIRD_PARTY_NOTICES.md and docs/tts_licensing.md. The defaults (FLUX.2 Klein, LTX-Video, ACE-Step, and the OpenF5 narration model) are commercial-friendly; the original F5-TTS narration weights are offered only as an opt-in non-commercial preview, Qwen-Image 2.1 is an opt-in research-only image engine, and the opt-in MiniMax engines carry their own community licenses. Review the notices before monetizing.

"Stephen Spielbot" is a playful name and is not affiliated with, endorsed by, or connected to Steven Spielberg or any of his companies.