hook-smith:带确定性幻觉红线过滤的冷邮件生成工具
yashsabbavarapu/hook-smith
hook-smith 是一款冷邮件生成工具,用不调用 LLM 的确定性校验器逐句复核生成内容,凡输入记录中不存在的金额、百分比、人数、日期、技术或实体,以及归属错误的真实数据,都会被标记 [REDLINED: Unverified claim] 并判定该 hook 失败。
Signal-to-copy outreach synthesizer with a deterministic hallucination redline filter.
Most AI SDR tools invent case studies, metrics and funding events, which burns domain
reputation. hook-smith ingests a structured lea lead-enrichment record, generates
personalized cold email angles, and then re-parses every generated sentence with a
deterministic validator that never calls an LLM. Any dollar figure, percentage,
headcount, date, technology or named entity that is not present in the input record (or
that is real but attributed to the wrong signal) is tagged [REDLINED: Unverified claim],
and the hook fails.
The generator is replaceable. The redline filter is the product.
Setup
python3 -m venv .venv && .venv/bin/pip install -e ".[dev]" export GEMINI_API_KEY=... # optional; without it the offline grounded composer is used
Usage
Generate for one lead, several, or a whole directory, leads run concurrently:
hooksmith generate --input fixtures/sample_lead.json hooksmith generate --input fixtures/ --concurrency 8 --out results/ --strict hooksmith generate --input fixtures/sample_lead.json --count 5 --json
Redline a draft a human (or another tool) wrote. No API key needed. This is the guardrail on its own:
hooksmith validate --input fixtures/sample_lead.json \
--draft fixtures/hallucinated_draft.txt --subject "Your Series C" --strict--strict exits 1 if anything is redlined, so it drops into CI or a send-gate.
Exit codes: 0 clean, 1 redlined under --strict, 2 a lead failed to generate.
A bad lead in a batch is reported and skipped; the rest still run.
Every verified claim carries the URL that backs it:
[Funding Growth] PASS
Subject: Linear's $35,000,000 Series B round
Linear raised a $35,000,000 Series B round with backing from Accel and Sequoia Capital.
✓ $35,000,000 https://linear.app/blog/series-b
✓ Series B https://linear.app/blog/series-b
✓ Accel https://linear.app/blog/series-b
How the redline engine works
validator.py runs the same claim extractor over both the draft and the lead record,
then checks set membership. Claims are canonicalised, so $35M, $35,000,000 and
$35 million all reduce to 35000000.00 and compare equal.
| Claim type | Caught example |
|---|---|
| money | $50M when the record says $35,000,000 |
| percent | a 40% drop in cycle time |
| funding round | Series C when the record says Series B |
| year / date | 2019 when the record says 2022-06-14 |
| headcount | 340 engineers, 12 to 340 engineers, 90 days |
| technology | Kubernetes, Next.js when not in the stack |
| named entity | Andreessen Horowitz, Datadog, fabricated investors and logos |
| misattribution | 4 open Infrastructure Engineer roles when 4 belongs to a different role |
| citation | a cited_signals entry that does not resolve to the record |
Claims are bound to the signal they name
A number is only true of something. The Linear fixture holds 4 Product Engineer roles
and 2 Infrastructure roles, so 4 open Infrastructure Engineer roles is false even though
4 appears in the record. Each funding round and job posting is a SignalScope with its
own facts and the claim kinds it owns; when a sentence names a signal, claims of those
kinds are checked against that signal alone. Sentences that name no signal fall back to
the whole record. Misattribution is reported separately from invention, because "real
number, wrong subject" is the subtler failure.
The vocabulary adapts to the lead
Technology detection uses hooksmith/data/tech_lexicon.json plus the lead's own
tech_stack, so a tool the bundled lexicon has never heard of is still validated
correctly. Extend the lexicon with HOOKSMITH_TECH_LEXICON=/path/to/file.json.
The model is resolved at runtime
There is no pinned model. The client lists models that currently support
generateContent, picks the newest general-purpose Flash, and keeps the runners-up.
It then handles three real failures seen against the live API:
- Retirement. A model can still be listed after
generateContentis retired for it (gemini-2.0-flashandgemini-2.5-flashboth 404 this way). The 404 body names the replacement, so the client follows it. - Quota. Limits are per model, so a 429 falls through to the next candidate.
- Load, transient 5xx is retried with backoff.
HOOKSMITH_MODEL pins a specific model and skips discovery.
Known limitations
Stated plainly, because a guardrail you trust incorrectly is worse than none:
- Sentence-initial names are skipped. The first word of a sentence is capitalised by
grammar, so it cannot be judged as a name.
Gartner named you a leader.slips through, whileYou were named by Gartner.is caught. - A sentence naming two signals accepts either signal's numbers. Attribution narrows to the union of the signals named, not to clause-level parsing.
- Ambiguous lowercase brands are not flagged.
stripeandfigmaonly count as claims when capitalised, to avoid false positives on ordinary English. - Semantic hallucinations are out of scope. "Your team is drowning in tickets" asserts no checkable quantity. This filter verifies facts, not insinuation.
Development
pytest -v # 72 tests ruff check .
CI runs the suite on Python 3.11–3.13, plus the CLI against the fixtures with --strict,
so the guardrail itself is exercised on every push.
来源:GitHub · github.com