AI-Job-Email-Sender：基于 Neo4j 知识图谱的求职邮件自动生成流水线

Job Send Crawl

AI-assisted job outreach automation for developers. Job Send Crawl parses a candidate resume, analyzes GitHub projects, imports company/job records, builds a Neo4j knowledge graph, ranks the best project-company matches, and generates personalized application emails.

What This Project Does

Job applications are repetitive, but good outreach still needs context. This project turns candidate data and company data into an explainable matching pipeline:

1. Parse a resume into structured candidate data.
2. Discover and summarize GitHub projects.
3. Import companies and job openings from CSV, JSON, Excel, or Google Sheets.
4. Build a knowledge graph of candidates, projects, technologies, capabilities, companies, roles, and jobs.
5. Index projects and jobs in a vector database.
6. Rank candidate projects against each company using graph, embedding, and LLM scores.
7. Generate personalized email drafts with project links and company-specific reasoning.
8. Queue or dry-run email sending from a frontend cockpit or CLI.

Highlights

• End-to-end pipeline: resume parsing, GitHub parsing, graph build, matching, draft generation, and mail processing.
• Knowledge graph matching: Neo4j stores explainable relationships between projects, capabilities, domains, companies, and job requirements.
• Hybrid ranking: combines graph overlap, vector similarity, and LLM scoring.
• Company import workflow: preview and validate CSV, JSON, Excel, or Google Sheet company data before launching a run.
• Frontend cockpit: React/Vite UI for creating runs, monitoring pipeline progress, viewing artifacts, reviewing matches, drafts, companies, and mail queue output.
• Provider-aware LLM routing: Groq, OpenAI, and Gemini providers with fallback behavior.
• Operational services: Redis for draft queues/rate limiting, Neo4j for graph storage, and Qdrant for vector search.
• CLI-first backend: every major module has a thin CLI entrypoint for local automation and debugging.

Architecture

flowchart LR
    Resume[Resume file] --> ParseResume[Resume parser]
    ParseResume --> GitHub[GitHub parser]
    Companies[Company spreadsheet or sheet URL] --> Import[Company import]

    ParseResume --> Graph[Knowledge graph builder]
    GitHub --> Graph
    Import --> Graph

    GitHub --> Vectors[Vector index]
    Import --> Vectors

    Graph --> Ranker[Hybrid project ranker]
    Vectors --> Ranker
    Ranker --> Drafts[Email draft generator]
    Drafts --> Queue[Redis mail queue]
    Queue --> Mail[SMTP or dry-run mail processing]

    API[FastAPI] --> RunStore[Run store and artifacts]
    RunStore --> Graph
    UI[React frontend] --> API

Loading

Core packages:

Tech Stack

Quick Start — Docker (recommended)

Run the entire stack (frontend, FastAPI backend, Redis, Neo4j, and Qdrant) with Docker Compose. You need Docker Desktop or Docker Engine with the Compose plugin.

1. Configure the application

From the repository root:

cp .env.example .env

Edit .env with the API keys you intend to use (Groq, OpenAI, or Gemini), plus a GITHUB_TOKEN for GitHub parsing. Configure mail credentials only when you are ready to send real email. The Compose configuration supplies local Redis, Qdrant, and Neo4j connection addresses for the backend. Neo4j defaults to neo4j/changeme for a new local volume; set NEO4J_PASSWORD in .env and keep it consistent with your Compose environment for other setups. Do not commit .env.

2. Build and start every service

docker compose up --build -d
docker compose ps

The first build may take a few minutes. Confirm that the backend and frontend are running:

docker compose logs --tail=50 backend frontend

The backend should log Application startup complete. If a service exits, inspect its logs with docker compose logs <service>.

3. Open the portal

Visit http://localhost:8080. The frontend is a production Vite build served by Nginx, which forwards requests under /api/ to the FastAPI backend inside the Compose network. No separate npm run dev or local Uvicorn process is needed.

4. Use the portal step by step

1. Settings / readiness: Open Settings and check provider/service readiness. Add missing LLM or GitHub credentials to the root .env, then recreate the backend: docker compose up -d --force-recreate backend. You can also inspect http://localhost:8000/api/v1/system/status.
2. Create a run: Open New Run. Supply the requested candidate/resume information and GitHub details. Select the relevant pipeline options; prefer dry-run while testing.
3. Import companies: In the new-run workflow, provide your company/job data as CSV, JSON, Excel, or a supported Google Sheet URL. Use the import preview to validate the records and correct missing or malformed fields before proceeding.
4. Launch and monitor: Start the run, then use Dashboard or Run Detail to monitor each pipeline step. The pipeline parses the resume and GitHub projects, builds graph/vector data, ranks project-company matches, and generates drafts. Review any reported errors before retrying or resuming.
5. Review candidate and companies: Open Candidate and Companies to inspect the extracted candidate information and imported company records.
6. Review matches: Open Matches to see the project-to-company recommendations. Check that the suggested projects and links are relevant.
7. Review email drafts: Open Drafts and verify the recipient, company, claims, project links, and wording. The current UI may not expose all draft-editing capabilities; edit/review through the API if needed.
8. Test the queue safely: Open Queue to review queue/mail processing output. Use dry-run first, which avoids sending real emails. Only enable real delivery after validating recipients, drafts, and mail credentials.

Important: Some pages and controls are still being improved, and a successful container startup does not guarantee that every pipeline integration is configured. LLM features require valid provider keys; live email sending requires valid mail configuration.

5. Stop or troubleshoot

docker compose down                 # stop containers; preserve named volumes
docker compose up --build -d        # rebuild and restart after code changes
docker compose logs -f backend      # inspect backend logs
docker compose ps                   # inspect service status and published ports

Avoid docker compose down -v unless you intentionally want to delete the local Redis, Neo4j, and Qdrant data volumes.

Where everything runs after Docker setup

All these ports are published to your machine by the default docker-compose.yml. They are intended for local development, not for exposure to the public internet without additional security controls.

Alternative: run backend and frontend outside Docker

For local hot-reload development, install the backend dependencies and start the infrastructure separately, then run Uvicorn and Vite on your host. The Docker quick start above is the preferred option when you simply want the whole application running.

./setup_uv_env.sh
cp .env.example .env
.venv/bin/python -m uvicorn app.api.main:app --reload

In a separate terminal:

cd frontend
npm install
cp .env.example .env.local
npm run dev

For a host-run Vite frontend, set VITE_API_BASE_URL=http://localhost:8000/api/v1 in frontend/.env.local. This alternative requires Redis, Neo4j, and Qdrant to be running and reachable.

Open the portal at http://localhost:5173 when using the Vite development server. If you start the containerized app with docker compose -f docker-compose.yml -f docker-compose.app.yml up -d --build, open http://localhost:8080 instead.

How to Use the Portal

1. Check system readiness

Open the portal and use Dashboard to confirm that Redis, Neo4j, and Qdrant are available. The status badges also show whether an LLM provider, GitHub token, and mail provider are configured. You can run safely without sending mail by keeping dry run enabled.

For endpoint-level details, open Settings. Secret values are never displayed; the page only reports masked readiness and configured service endpoints.

2. Start a new run and import inputs

Select New Run in the sidebar, then complete the four wizard stages:

1. Import: upload a PDF, DOCX, or text resume. Add companies from an Excel, CSV, or JSON file, or switch to Google Sheet URL and paste a shared sheet link. Adjust the header row, sheet names, or row limit when needed, then select Parse companies.
2. Select: review the normalized company rows and select one or more valid companies. Rows with validation issues cannot be selected until the source data is fixed.
3. Options: add the recipient email and optional job URL, then choose repository, company, and match limits. Leave Dry run checked for the first run. Use Do not enqueue if you only want generated artifacts.
4. Review: verify the selected companies and options, then select Start run.

3. Monitor and review the results

After the run starts, the portal opens its run detail page. Follow the step status and logs until the pipeline finishes. The run selector in the top bar lets you switch between previous runs.

Use the sidebar workspaces to inspect the generated output:

• Candidate and Resume Builder: review parsed candidate data and resume content.
• Companies: inspect the selected and normalized company records.
• Matches: compare ranked project-company matches and their scores.
• Drafts: expand each company to review the recipient, subject, and generated email body. Select Enqueue All only after reviewing the drafts.
• Queue: inspect queued results and select Process dry run to test mail processing without sending real email.

The candidate and company workspaces turn the parsed artifacts into reviewable records before you act on the generated outreach.

Expand a draft to verify its recipient, subject, body, company, and status before enqueueing it.

Finally, confirm that a test result is marked dry_run on the queue page. This verifies mail processing without delivering a message.

To send real email, configure a mail provider in .env, restart the backend, create a run with Dry run disabled, and review every draft and recipient before processing the queue.

End-to-End Pipeline

Run the full pipeline from the repository root:

./scripts/run_pipeline.sh --dry-run

Resume from a specific step:

./scripts/run_pipeline.sh --from-step 5 --dry-run

Useful flags:

Pipeline outputs are written as JSON artifacts:

API Overview

The FastAPI app currently exposes the run-oriented cockpit API:

CLI Reference

All CLIs use cli/bootstrap.py for import setup. Run from the repo root with .venv/bin/python -m ....

Company import

.venv/bin/python -m cli.excel.parse_excel ./companies.xlsx \
  --output-file data/companies_sheet.json

Resume parsing

.venv/bin/python -m cli.resume.parse_resume ./data/resume.pdf \
  --output-file data/parse_resume.json

Use --text-only to extract text without structured LLM parsing.

GitHub parsing

.venv/bin/python -m cli.github.parse_github ./data/resume.pdf \
  --output-file data/github_projects_resume.json \
  --max-repos 50

Set GITHUB_TOKEN for reliable GitHub API access. Use --readme-only to skip LLM extraction.

Knowledge graph

.venv/bin/python -m cli.graph.build_graph \
  --resume data/parse_resume.json \
  --github data/github_projects_resume.json \
  --companies data/companies_sheet.json \
  --clear \
  --output-file data/graph_build.json

--clear removes all Neo4j nodes before rebuild.

Matching

.venv/bin/python -m cli.matching.rank_projects \
  --companies data/companies_sheet.json \
  --company "Acme" \
  --candidate-id candidate:github_username \
  --output-file data/matches.json

Read the candidate ID from graph_build.json at candidate.metadata.candidate_id.

Email drafts

.venv/bin/python -m cli.emails.generate_draft \
  --resume data/parse_resume.json \
  --matches data/matches.json \
  --github data/github_projects_resume.json \
  --companies data/companies_sheet.json \
  --company "Acme" \
  --recipient-email hr@example.com \
  --output-file data/drafts.json

Generated drafts append a GitHub project links section. Deployed links are preferred when available.

Mail queue

.venv/bin/python -m cli.mail.process_queue --dry-run \
  --output-file data/mail_queue_result.json

SMTP credentials are required only when sending real email, not for --dry-run.

Programmatic Usage

from pathlib import Path

from pipeline import ApplicationPipelineBuilder, PipelineOptions, PipelineStep

pipeline = (
    ApplicationPipelineBuilder(project_root=Path("."))
    .with_resume("data/resume.pdf")
    .with_companies("data/companies_sheet.json")
    .with_options(PipelineOptions(dry_run=True, from_step=1))
    .build()
)

result = pipeline.run()
print(result.context.matches)

Run a custom subset of steps:

result = pipeline.run(
    steps=(
        PipelineStep.PARSE_RESUME,
        PipelineStep.PARSE_GITHUB,
        PipelineStep.BUILD_GRAPH,
    )
)

Configuration

Settings load from environment variables and .env through app/core/settings.py.

Provider order for the LLM router is Groq keys 1-4, OpenAI, then Gemini. On rate limits, the router pauses that provider and tries the next configured provider.

Testing

Backend:

.venv/bin/python -m pytest

Focused suites:

.venv/bin/python -m pytest tests/api
.venv/bin/python -m pytest tests/pipeline
.venv/bin/python -m pytest tests/modules/graph
.venv/bin/python -m pytest tests/modules/matching
.venv/bin/python -m pytest tests/modules/emails

Frontend:

cd frontend
npm run test
npm run build

Documentation

• High-level design: doc/HLD.md
• Graph architecture: app/modules/graph/doc/ARCHITECTURE.md
• Graph low-level design and zero-score debugging: app/modules/graph/doc/LLD.md

Project Layout

pipeline/         ApplicationPipelineBuilder and step handlers
app/
  api/            FastAPI app, routes, schemas, and API services
  core/           settings, logging, exceptions, constants
  modules/        excel, resume, github, graph, vector, matching, emails, mail, llm
  celery/         Celery app and task registration
  redis/          email draft queue and rate limiting
cli/              thin CLI entrypoints
frontend/         React/Vite cockpit
prompts/          shared LLM prompt constants
scripts/          run_pipeline.sh and wait_for_services.sh
tests/            backend and frontend-adjacent test suites
doc/              design docs and README assets

Current Frontend Follow-Ups

The cockpit covers the main run workflow today. Known next improvements:

• Add draft editing UI on top of PUT /runs/{run_id}/drafts.
• Align draft API client types with the backend's per-company draft dictionary response.
• Replace polling with the existing /runs/{run_id}/events stream.
• Add richer graph/match artifact views, including graph paths and zero-score diagnostics.
• Add a run picker for shared pages such as Candidate, Companies, Matches, Drafts, and Queue.

Contributing

Contributions are welcome. Start with CONTRIBUTING.md for development setup, test commands, branch guidance, and pull request expectations.

Before starting larger work, check the existing issues and pull requests to avoid duplicate effort. Issues labeled good first issue or help wanted are intended to be easier entry points for external contributors.

Please follow the Code of Conduct in project spaces.

Security

Do not report vulnerabilities, credentials, personal data exposure, or email-sending security issues through a normal public bug report. Follow SECURITY.md for responsible disclosure guidance.

Support the project

If Job Send Crawl is useful to you, you can support ongoing development through the repository's Sponsor button once GitHub Sponsors is enabled for the maintainer account.

Contributions, bug reports, documentation improvements, and feedback are equally welcome.

License

Licensed under the Apache License 2.0.

Status

This project is under active development. The safest operating mode is --dry-run until mail credentials and queue behavior are verified for your environment.