shubhamcodess/everything-tech-newsletter：用 Claude Code 自动生成每日科技报纸

A daily tech newspaper with no human editor. Python pulls fresh articles from 47 free sources, groups duplicate coverage, and fetches the full text of the stories that matter. A scheduled Claude Code routine picks, ranks, and writes each one up from the actual article: a headline, a dek, a few real paragraphs, and takeaways. GitHub Pages serves it. No server, no database, no API key.

How it works

┌──────────────────────────────────────────────────────┐
│                      47 sources                      │
│              RSS · APIs · HTML scrapes               │
└──────────────────────────────────────────────────────┘
                            │  fetch_all.py
                            ▼
┌──────────────────────────────────────────────────────┐
│  build_candidates.py -> data/candidates.md (~14K tk) │
│  drop stale / seen / junk · cluster same-story news  │
└──────────────────────────────────────────────────────┘
                            │  routine picks + ranks
                            ▼
┌──────────────────────────────────────────────────────┐
│  fetch_articles.py -> data/articles.md   (~19K tk)   │
│  full article text, fallback to other outlets        │
└──────────────────────────────────────────────────────┘
                            │  routine writes data/draft.md
                            ▼
┌──────────────────────────────────────────────────────┐
│  publish_edition.py                                  │
│  validate prose · fill links/authors/images          │
│  -> editions/<date>/edition.md  (the durable record) │
└──────────────────────────────────────────────────────┘
                            │  build_site.py
                            ▼
┌──────────────────────────────────────────────────────┐
│  docs/data/*.json + feed.xml  ──►  one template      │
│  (docs/assets) renders today and every archive date  │
└──────────────────────────────────────────────────────┘
                            │  commit + push (or PR -> auto-merge)
                            ▼
┌──────────────────────────────────────────────────────┐
│          GitHub Pages  (serves main:/docs)           │
└──────────────────────────────────────────────────────┘

Scripts do everything mechanical; the LLM only picks, ranks, and writes. It never writes HTML: each edition is a Markdown file, and one client-side template draws every date from it. Change the design once and every edition, today's and archived, changes with it. Its instructions live in CLAUDE.md, versioned like any other code.

Written from the source. Full article text, not a feed blurb, and a validator rejects copied sentences, filler, and cut-off prose.
Deduped. One story covered by five outlets shows once, crediting all five.
Ranked. The lead, must-reads, and the wire, plus a 60-second Brief.
Readable. Night and day themes, topic and signal filters, keyboard shortcuts (?), and a live ticker.
Portable. Every edition also comes as Markdown, JSON and RSS.
Self-pruning archive. The last 30 days (archive_days in settings); older editions stay in git history.

Fork it and make it yours

You'll need a Claude plan that includes routines (Pro, Max, Team, or Enterprise) and a GitHub account.

1. Fork and reset. Fork this repo, clone it, then start clean:

./scripts/reset_for_fork.sh    # empty archive, no remembered stories

2. Make it yours. Name, tagline, curator links and colours live in config/site.json. Topics go in config/interests.json, and edition size in config/settings.json. Then rebuild the site data:

python scripts/build_site.py

3. Turn on GitHub Pages. Repo Settings → Pages → Deploy from a branch → main / /docs. (Pages docs)

4. Two repo settings for the auto-merge handoff. The routine can't always push straight to a protected main, so it opens a PR and auto-merge-routine.yml merges it.

Settings → General → Pull Requests → Allow auto-merge
Settings → Branches → add a rule for main → Require status checks → validate (protected branches)

5. Create the routine at claude.ai/code/routines/new:

Field	Value
Prompt	the text in `ROUTINE_PROMPT.md`
Model	Sonnet for the best writing; Haiku works
Repository	your fork
Environment	Network access → Full (it reads articles on any site), or Custom with the list in `SETUP.md` (why)
Trigger	Schedule → Daily → your time
Connectors	none

6. Run it. Click Run now, watch the session, then check your Pages URL. Details and troubleshooting: SETUP.md.

Configuration

Edit, commit, push. The next run picks it up.

File	Controls
`config/site.json`	Render config: name, tagline, curator links, theme colours, features (run `build_site.py` after)
`config/interests.json`	Topics, and what each covers
`config/settings.json`	Stories per edition, must-read cap, lookback, token budgets, archive days
`config/sources.json`	Add, remove, or disable sources; RSS needs only a `feed_url`
`docs/assets/`	The one template: every edition renders through it
`CLAUDE.md`	The routine's step-by-step pipeline and writing rules

Project structure

.
├── CLAUDE.md                       # the routine's pipeline + writing rules
├── ROUTINE_PROMPT.md               # the prompt pasted into the routine
├── SETUP.md                        # one-time setup + troubleshooting
├── .github/workflows/
│   ├── ci.yml                      # validates config, fetchers, editions, site data
│   └── auto-merge-routine.yml      # merges the routine's claude/* PRs once CI passes
├── config/
│   ├── site.json                   # render config for the whole site
│   ├── interests.json              # topics
│   ├── settings.json               # edition size, budgets, windows
│   └── sources.json                # the 47-source registry
├── scripts/
│   ├── fetch_all.py                # runs every fetcher -> data/raw_latest.json
│   ├── fetch_*.py                  # RSS, HN, Reddit, arXiv, Dev.to, GitHub Trending, TLDR
│   ├── build_candidates.py         # filter + cluster -> data/candidates.md
│   ├── fetch_articles.py           # full text of the picks -> data/articles.md
│   ├── publish_edition.py          # validate draft -> editions/<date>/
│   ├── build_site.py               # editions -> docs/data, feed.xml, latest.*
│   ├── edition_md.py               # the edition Markdown format
│   ├── edition_schema.py           # prose + structure validation
│   ├── validate_editions.py        # CI check
│   └── reset_for_fork.sh           # clean slate for a fresh fork
├── editions/
│   └── YYYY-MM-DD/
│       ├── edition.md              # the day's paper, self-contained
│       ├── sources.md              # full text the stories were written from
│       └── candidates.md           # everything that was considered
├── data/
│   └── seen.json                   # recently featured stories (no repeats)
├── assets/
│   └── preview.png                 # README banner
└── docs/                           # GitHub Pages source (branch deploy)
    ├── index.html                  # app shell
    ├── assets/                     # app.css + app.js: the one template
    ├── data/                       # site.json, index.json, <date>.json
    ├── feed.xml                    # RSS
    └── latest.md / latest.json     # today's edition, machine-readable

$ curated_by Shubham Prakash