跳到正文
GitHub·· 7 天前AI 评分26

bookmark-agent:面向 AI 智能体的书签采集 REST API

tuanphungcz/bookmark-agent

AI 导读

bookmark-agent 是一个 API 优先的书签采集服务,保存链接后自动在后台抓取页面内容,并以 HTML、Markdown 和纯文本三种格式存储快照,支持重新抓取更新、URL 去重和浏览器 HTML 书签导入。

正文

Bookmark ingestion API. Save links, fetch their content automatically, and access everything through a clean REST API. Runs on SQLite -- no external database required.

Live instance: bookmark-agent.com

Why

Most bookmark tools lock your data in a browser or a SaaS. bookmark-agent gives you a plain API so anything can save and read bookmarks -- scripts, CLI tools, browser extensions, AI agents, or your own frontend.

Features

  • API-first -- every feature is an HTTP endpoint, no UI required
  • Automatic content snapshots -- pages are fetched in the background and stored as HTML, Markdown, and plain text
  • Versioned snapshots -- re-fetch a page anytime to capture updates
  • Deduplication -- the same URL is stored once, even across multiple imports
  • HTML import -- drop in a browser bookmark export and everything gets ingested
  • X/Twitter support -- tweets are fetched via FixTweet, no API key needed
  • API keys -- bootstrap a personal key, use Authorization: Bearer everywhere
  • SQLite-backed -- single-file database, zero infrastructure
  • Background task queue -- fetches and imports run async without blocking requests
  • Rate limiting -- built-in per-user rate limits for auth and write endpoints
  • AI-friendly -- serves a machine-readable skill file so AI agents can discover and use the API

Quick start

pnpm install
cp .env.example .env
pnpm dev

API runs on http://localhost:3004.

Docker

cp .env.example .env
docker compose up --build

Data persists in the bookmark-agent-data Docker volume. Health check: GET /api/health.

API key flow

API keys are the recommended auth method for programmatic access.

# 1. Sign in (once)
curl -s -c /tmp/ba-cookies.txt -X POST "https://bookmark-agent.com/api/auth/sign-in/email" \
  -H "Content-Type: application/json" \
  -d '{"email":"you@example.com","password":"your-password"}'

# 2. Bootstrap your API key (once)
curl -s -b /tmp/ba-cookies.txt -X POST "https://bookmark-agent.com/api/account/api-key/bootstrap"
# -> returns { "token": "ba_..." } -- save this, it's shown only once

# 3. Use the key for everything else
curl -s -H "Authorization: Bearer ba_..." "https://bookmark-agent.com/api/bookmarks"

# 4. Rotate if compromised (requires session, not the old key)
curl -s -b /tmp/ba-cookies.txt -X POST "https://bookmark-agent.com/api/account/api-key/roll"

Endpoints

Method Path Auth Description
ALL /api/auth/* -- Sign up, sign in, sign out (better-auth)
GET /api/account/api-key session Get current API key metadata
POST /api/account/api-key/bootstrap session Create first API key
POST /api/account/api-key/roll session Rotate API key
POST /api/bookmarks bearer/session Create a bookmark
GET /api/bookmarks bearer/session List bookmarks
GET /api/bookmarks/:id bearer/session Get one bookmark
PATCH /api/bookmarks/:id bearer/session Update bookmark
DELETE /api/bookmarks/:id bearer/session Delete bookmark
POST /api/bookmarks/:id/refetch bearer/session Re-fetch page content
POST /api/imports/html bearer/session Import browser HTML export
GET /api/imports bearer/session List imports
GET /api/imports/:id bearer/session Get import status
GET /api/health none Health check

AI integration

The root URL serves a skill file that describes every endpoint, auth flow, and parameter. Point your AI agent at https://bookmark-agent.com/skill.md and it can start using the API immediately.

A Claude Code skill is also included at .claude/skills/bookmark-agent.md for users who clone this repo.

Security

What's in place

  • API key secrets are SHA-256 hashed before storage; lookup uses a prefix, verification uses constant-time comparison
  • Passwords are handled by better-auth (bcrypt)
  • All input is validated with Zod schemas with strict length and type constraints
  • All database queries use Drizzle ORM with parameterized statements -- no raw SQL string concatenation
  • External URL fetches have SSRF protection (private IP blocking, redirect limits, timeouts, response size caps)
  • Session cookies use HttpOnly, SameSite=Lax, and Secure (in production)
  • CORS is locked to a single configured origin
  • Rate limiting on auth and write endpoints

Known limitations

  • In-memory rate limiter -- resets on restart, doesn't share across instances. For production behind a load balancer, add edge-level rate limiting
  • SSRF DNS rebinding -- the private-IP check runs before fetch(), which re-resolves DNS independently. A DNS rebinding attack could bypass the check. Mitigate with egress firewall rules blocking RFC 1918 ranges
  • Rate limit headers are trustable only behind a proxy -- X-Forwarded-For and similar headers are used for rate limit keying. Deploy behind a reverse proxy (Cloudflare, Traefik, nginx) that sets these headers authoritatively
  • No email verification -- signups don't require email confirmation. Use BOOKMARK_AGENT_SIGNUP_ALLOWLIST to restrict who can register

Environment variables

Variable Purpose
BETTER_AUTH_SECRET Required. Encryption key for auth. Use a random 32+ character string
BOOKMARK_AGENT_SIGNUP_ALLOWLIST Comma-separated emails allowed to register
BOOKMARK_AGENT_AUTH_RATE_LIMIT_MAX Max auth requests per window (default: 20)
BOOKMARK_AGENT_AUTH_RATE_LIMIT_WINDOW_MS Auth rate limit window (default: 60000)
BOOKMARK_AGENT_WRITE_RATE_LIMIT_MAX Max write requests per window (default: 120)
BOOKMARK_AGENT_WRITE_RATE_LIMIT_WINDOW_MS Write rate limit window (default: 60000)

Architecture

apps/api          Hono HTTP server, routes, background workers
packages/auth     better-auth integration, session + API key middleware
packages/db-core  Drizzle schema, migrations, SQLite client
packages/server   Shared Hono utilities, CORS, rate limiting, static serving
packages/env      Environment variable validation
packages/cron     Lightweight cron scheduler
packages/task-queue  SQLite-backed async task queue

Development

pnpm lint        # Biome
pnpm typecheck   # TypeScript
pnpm test        # Vitest

CI runs the same checks on every push and PR.

Deployment

Dokploy

A deploy.json is included for Dokploy deployments:

  • Persistent volume at /app/data (SQLite lives here)
  • Set BETTER_AUTH_SECRET in your environment
  • Auto-deploy on push is supported via GitHub webhook

Manual Docker

docker build -t bookmark-agent .
docker run -p 3004:3004 -v bookmark-data:/app/data \
  -e BETTER_AUTH_SECRET=$(openssl rand -base64 32) \
  -e NODE_ENV=production \
  bookmark-agent

License

GNU GPL v3.0

来源:GitHub · github.com