bookmark-agent:面向 AI 智能体的书签采集 REST API
tuanphungcz/bookmark-agent
bookmark-agent 是一个 API 优先的书签采集服务,保存链接后自动在后台抓取页面内容,并以 HTML、Markdown 和纯文本三种格式存储快照,支持重新抓取更新、URL 去重和浏览器 HTML 书签导入。
Bookmark ingestion API. Save links, fetch their content automatically, and access everything through a clean REST API. Runs on SQLite -- no external database required.
Live instance: bookmark-agent.com
Why
Most bookmark tools lock your data in a browser or a SaaS. bookmark-agent gives you a plain API so anything can save and read bookmarks -- scripts, CLI tools, browser extensions, AI agents, or your own frontend.
Features
- API-first -- every feature is an HTTP endpoint, no UI required
- Automatic content snapshots -- pages are fetched in the background and stored as HTML, Markdown, and plain text
- Versioned snapshots -- re-fetch a page anytime to capture updates
- Deduplication -- the same URL is stored once, even across multiple imports
- HTML import -- drop in a browser bookmark export and everything gets ingested
- X/Twitter support -- tweets are fetched via FixTweet, no API key needed
- API keys -- bootstrap a personal key, use
Authorization: Bearereverywhere - SQLite-backed -- single-file database, zero infrastructure
- Background task queue -- fetches and imports run async without blocking requests
- Rate limiting -- built-in per-user rate limits for auth and write endpoints
- AI-friendly -- serves a machine-readable skill file so AI agents can discover and use the API
Quick start
pnpm install cp .env.example .env pnpm dev
API runs on http://localhost:3004.
Docker
cp .env.example .env docker compose up --build
Data persists in the bookmark-agent-data Docker volume. Health check: GET /api/health.
API key flow
API keys are the recommended auth method for programmatic access.
# 1. Sign in (once) curl -s -c /tmp/ba-cookies.txt -X POST "https://bookmark-agent.com/api/auth/sign-in/email" \ -H "Content-Type: application/json" \ -d '{"email":"you@example.com","password":"your-password"}' # 2. Bootstrap your API key (once) curl -s -b /tmp/ba-cookies.txt -X POST "https://bookmark-agent.com/api/account/api-key/bootstrap" # -> returns { "token": "ba_..." } -- save this, it's shown only once # 3. Use the key for everything else curl -s -H "Authorization: Bearer ba_..." "https://bookmark-agent.com/api/bookmarks" # 4. Rotate if compromised (requires session, not the old key) curl -s -b /tmp/ba-cookies.txt -X POST "https://bookmark-agent.com/api/account/api-key/roll"
Endpoints
| Method | Path | Auth | Description |
|---|---|---|---|
ALL |
/api/auth/* |
-- | Sign up, sign in, sign out (better-auth) |
GET |
/api/account/api-key |
session | Get current API key metadata |
POST |
/api/account/api-key/bootstrap |
session | Create first API key |
POST |
/api/account/api-key/roll |
session | Rotate API key |
POST |
/api/bookmarks |
bearer/session | Create a bookmark |
GET |
/api/bookmarks |
bearer/session | List bookmarks |
GET |
/api/bookmarks/:id |
bearer/session | Get one bookmark |
PATCH |
/api/bookmarks/:id |
bearer/session | Update bookmark |
DELETE |
/api/bookmarks/:id |
bearer/session | Delete bookmark |
POST |
/api/bookmarks/:id/refetch |
bearer/session | Re-fetch page content |
POST |
/api/imports/html |
bearer/session | Import browser HTML export |
GET |
/api/imports |
bearer/session | List imports |
GET |
/api/imports/:id |
bearer/session | Get import status |
GET |
/api/health |
none | Health check |
AI integration
The root URL serves a skill file that describes every endpoint, auth flow, and parameter. Point your AI agent at https://bookmark-agent.com/skill.md and it can start using the API immediately.
A Claude Code skill is also included at .claude/skills/bookmark-agent.md for users who clone this repo.
Security
What's in place
- API key secrets are SHA-256 hashed before storage; lookup uses a prefix, verification uses constant-time comparison
- Passwords are handled by better-auth (bcrypt)
- All input is validated with Zod schemas with strict length and type constraints
- All database queries use Drizzle ORM with parameterized statements -- no raw SQL string concatenation
- External URL fetches have SSRF protection (private IP blocking, redirect limits, timeouts, response size caps)
- Session cookies use
HttpOnly,SameSite=Lax, andSecure(in production) - CORS is locked to a single configured origin
- Rate limiting on auth and write endpoints
Known limitations
- In-memory rate limiter -- resets on restart, doesn't share across instances. For production behind a load balancer, add edge-level rate limiting
- SSRF DNS rebinding -- the private-IP check runs before
fetch(), which re-resolves DNS independently. A DNS rebinding attack could bypass the check. Mitigate with egress firewall rules blocking RFC 1918 ranges - Rate limit headers are trustable only behind a proxy --
X-Forwarded-Forand similar headers are used for rate limit keying. Deploy behind a reverse proxy (Cloudflare, Traefik, nginx) that sets these headers authoritatively - No email verification -- signups don't require email confirmation. Use
BOOKMARK_AGENT_SIGNUP_ALLOWLISTto restrict who can register
Environment variables
| Variable | Purpose |
|---|---|
BETTER_AUTH_SECRET |
Required. Encryption key for auth. Use a random 32+ character string |
BOOKMARK_AGENT_SIGNUP_ALLOWLIST |
Comma-separated emails allowed to register |
BOOKMARK_AGENT_AUTH_RATE_LIMIT_MAX |
Max auth requests per window (default: 20) |
BOOKMARK_AGENT_AUTH_RATE_LIMIT_WINDOW_MS |
Auth rate limit window (default: 60000) |
BOOKMARK_AGENT_WRITE_RATE_LIMIT_MAX |
Max write requests per window (default: 120) |
BOOKMARK_AGENT_WRITE_RATE_LIMIT_WINDOW_MS |
Write rate limit window (default: 60000) |
Architecture
apps/api Hono HTTP server, routes, background workers
packages/auth better-auth integration, session + API key middleware
packages/db-core Drizzle schema, migrations, SQLite client
packages/server Shared Hono utilities, CORS, rate limiting, static serving
packages/env Environment variable validation
packages/cron Lightweight cron scheduler
packages/task-queue SQLite-backed async task queue
Development
pnpm lint # Biome pnpm typecheck # TypeScript pnpm test # Vitest
CI runs the same checks on every push and PR.
Deployment
Dokploy
A deploy.json is included for Dokploy deployments:
- Persistent volume at
/app/data(SQLite lives here) - Set
BETTER_AUTH_SECRETin your environment - Auto-deploy on push is supported via GitHub webhook
Manual Docker
docker build -t bookmark-agent . docker run -p 3004:3004 -v bookmark-data:/app/data \ -e BETTER_AUTH_SECRET=$(openssl rand -base64 32) \ -e NODE_ENV=production \ bookmark-agent
License
GNU GPL v3.0
来源:GitHub · github.com