PodLens：为播客生成 AI 上下文与实体卡片

PodLens: AI-Powered Context for Your Podcasts

PodLens is an AI-powered context engine for podcasts. It transcribes an episode, identifies the people, companies, places and technologies mentioned, and surfaces each one as a source card with a summary, map and image, timed to the moment it is spoken. It can also build a chronological timeline of the episode and answer questions about it, with citations that jump to the exact moment.

▶ Try the live demo: https://podlens.dhruvpathak.com

Built With

Category	Technologies
Languages
Frontend
Backend
Speech & audio
NLP & entities
LLM & AI
Data sources
Testing & quality
DevOps & tooling
Live mode (parked)

Also used: asyncio, python-multipart, typescript-eslint, eslint-plugin-react-hooks, Docker Compose, concurrently, and Python venv. Data is stored as JSON entity exports and timestamped .txt transcripts.

See it in Action

PodLens_Context.Gen.mp4

PodLens_Live.Context.mp4

Contents

• Built With
• Key Features
• How It Works
• AI Entity Review
• Episode Timeline
• Ask the Episode (LLM Q&A)
• Choosing Your Models: spaCy vs Claude vs OpenAI
• APIs and Services Used
• PodLens REST API
• Installation and Setup
• Running the App

Key Features

How It Works

 Upload audio ─▶ FFmpeg ─▶ Whisper ─▶ Transcript + segments
                                          │
                                          ▼
                              NER (spaCy or Claude) ─▶ Entities with timestamps
                                          │
              ┌───────────────────────────┼───────────────────────────┐
              ▼                           ▼                           ▼
   Source cards                   Episode timeline              Ask the episode
   Wikipedia · OSM · Unsplash     LangChain + OpenAI            LangChain + OpenAI
              │                           │                           │
              └──────────────▶ React dashboard synced to playback ◀───┘

AI Entity Review

Speech recognition and entity taggers make predictable mistakes on podcasts. A review step checks every entity against the episode before (and after) the source cards are built.

How it works

1. Before lookups: each unique entity is sent with the transcript lines it appears in and the opening of the episode. The model returns one verdict per entity (keep, fix, merge, drop) plus a disambiguated Wikipedia search query, using strict structured output.
2. Lookups: Wikipedia, maps and photos use the corrected names and search queries.
3. After lookups: the model compares each card's Wikipedia summary and photo description with how the episode uses the name, and strict mismatches are removed from the card.

The Entities panel shows the result ("checked by AI, 6 corrected, 5 removed"; hover for the full list). The review fails open: if the model is unavailable, cards are built from the original entities. Turn it off with ENTITY_REVIEW=false; demo builds accept --no-review.

🧭 Episode Timeline

One click turns the episode into a chronological timeline of every company, person, product, deal and event mentioned, with real-world dates where the speakers give them.

How it works

• The transcript is split into windows (TIMELINE_WINDOW_CHARS, default 24,000 characters) so long episodes fit the model's context.
• Windows are processed concurrently (TIMELINE_MAX_CONCURRENCY, default 4). Each window only receives the entities tagged inside it.
• LangChain's with_structured_output uses OpenAI strict JSON schema mode, so every event is guaranteed to match the schema; no regex parsing of model output.
• Results from all windows are merged, de-duplicated and clamped to valid timestamps. One failed window does not fail the whole timeline.
• In the UI, Create timeline lists every event in date order on a quiet date spine: the date on the left (in the accent colour), the event title and a Played at link to the moment it is mentioned on the right. Undated events come last; the model's description shows when you hover a title.

Event schema

type TimelineEvent = {
  id: string
  title: string
  description: string
  category: 'COMPANY' | 'PERSON' | 'PRODUCT' | 'TECHNOLOGY' | 'EVENT' | 'PLACE' | 'DEAL' | 'OTHER'
  date_label: string | null   // as stated: "1993", "late 1990s"
  year: number | null
  month: number | null
  mentioned_at_sec: number    // where it is first mentioned in the episode
  entities: string[]
  importance: 1 | 2 | 3       // passing mention, notable, central
}

Request

curl -X POST http://127.0.0.1:8000/api/timeline \
  -H "Content-Type: application/json" \
  -d '{
        "segments": [{"id": 0, "start": 12.0, "end": 23.0,
                      "text": "In 1966 researchers began studying lions in the Serengeti."}],
        "entities": [{"type": "PLACE", "text": "Serengeti", "start_sec": 13, "end_sec": 15, "chunk_id": 0}],
        "source_label": "Lions 101.mp3"
      }'

Response (abridged)

{
  "model": "gpt-5.4-mini",
  "windows": 1,
  "failed_windows": 0,
  "events": [
    {
      "id": "ev0",
      "title": "Long-term Serengeti lion study begins",
      "category": "EVENT",
      "date_label": "1966",
      "year": 1966,
      "mentioned_at_sec": 12.0,
      "entities": ["Serengeti"],
      "importance": 3
    }
  ]
}

Ask the Episode (LLM Q&A)

Ask anything about the episode and get an answer grounded only in the transcript, with citations you can click to hear the exact quote.

Hybrid context strategy

• Embeddings for a long episode are computed once and cached in memory, keyed by a fingerprint of the transcript, so follow-up questions are fast and cheap.
• The last 6 turns of the conversation are sent with each question, so follow-ups like "when did he say that?" work.
• Citation validation: every quote the model returns is checked against the transcript. Citations that do not match are dropped, and an answer whose citations all fail is flagged unverified: true.
• If the episode does not cover the question, the model says so (found: false) instead of guessing.

Request

curl -X POST http://127.0.0.1:8000/api/ask \
  -H "Content-Type: application/json" \
  -d '{
        "question": "Who wrote The Serengeti Lion?",
        "segments": [{"id": 2, "start": 24.0, "end": 35.0,
                      "text": "George Schaller published The Serengeti Lion in 1972."}],
        "history": []
      }'

Response

{
  "answer": "George Schaller, who published The Serengeti Lion in 1972.",
  "found": true,
  "citations": [{ "start_sec": 24.0, "quote": "George Schaller published The Serengeti Lion in 1972." }],
  "mode": "full",
  "model": "gpt-5.4-mini",
  "unverified": false
}

Choosing Your Models: spaCy vs Claude vs OpenAI

PodLens uses three model families, each for a different job:

OpenAI does not tag entities. It reviews what spaCy or Claude found (fix, merge, drop) and powers the timeline and Q&A.

Entity tagging: spaCy vs Claude

How the backend is chosen: ENTITY_BACKEND wins if set. If it is unset and ANTHROPIC_API_KEY is present, Claude is used; otherwise spaCy. The UI can force one with VITE_ENTITY_BACKEND.

Whisper model sizes

APIs and Services Used

Every external service is optional except Whisper and spaCy. Missing keys disable only the feature that needs them, and GET /api/health reports which features are active.

PodLens REST API

Interactive docs are available at http://127.0.0.1:8000/docs while the API is running.

Transcribe an episode

curl -X POST http://127.0.0.1:8000/api/transcribe \
  -F "audio=@episode.mp3" \
  -F "extract_entities=true" \
  -F "entity_backend=spacy"     # optional: spacy | claude

{
  "transcript": "Lions are the only truly social big cats...",
  "segments": [{ "id": 0, "start": 0.0, "end": 11.0, "text": "Lions are the only truly social big cats..." }],
  "saved_path": ".../server/transcripts/episode_20261008T221500Z.txt",
  "document": {
    "backend": "spacy",
    "entities": [{ "type": "PLACE", "text": "Serengeti", "start_sec": 13.0, "end_sec": 15.0, "chunk_id": 1 }]
  },
  "entity_error": null
}

Build source cards

curl -X POST http://127.0.0.1:8000/api/enrich-entities \
  -H "Content-Type: application/json" \
  -d '{"entities": [{"type": "PLACE", "text": "Serengeti", "start_sec": 13, "end_sec": 15, "chunk_id": 1}]}'

Check which features are on

curl http://127.0.0.1:8000/api/health
# {"ok":true,"model":"base","entity_backend":"spacy","unsplash_configured":true,
#  "openai_configured":true,"entity_review":true,"live_mode":false,"realtime_transcription":false}

Installation and Setup

Prerequisites

• Node.js 18+
• Python 3.9+
• FFmpeg on your PATH (brew install ffmpeg on macOS)

1. Clone and install the frontend

git clone https://github.com/dhruvpathak1/PodLens_AI_Context_Generator.git
cd PodLens_AI_Context_Generator
npm install

2. Set up the backend

The npm scripts expect the virtual environment at the project root (venv/).

python -m venv venv
venv/bin/python -m pip install -r server/requirements.txt   # Windows: venv\Scripts\python

3. Configure

cp .env.example .env

Then fill in what you need:

Running the App

Development

npm run dev

• Frontend: http://localhost:5173
• Backend API: http://127.0.0.1:8000 (docs at /docs)

The first transcription downloads the Whisper model, which can take a few minutes.

Checks

npm run check        # typecheck + lint + backend tests
npm run build        # production bundle in dist/

Backend tests need the dev requirements once: venv/bin/python -m pip install -r server/requirements-dev.txt.

🐳 Docker

docker compose up --build

Open http://localhost:8080. One container serves the built UI and the API; data persists in the podlens-data volume. See DOCKER.md for sharing and deployment.

Distributed under the MIT License. See LICENSE for more information.

Created with ❤️ for the podcasting community.