Buncrawl：基于 Bun 原生 API 的 AI 智能体网页数据 API

Buncrawl is a web data API for AI agents, built with native Bun APIs.

Tech Stack

• TypeScript - For type safety and improved developer experience
• TanStack Router - File-based routing with full type safety
• TailwindCSS - Utility-first CSS for rapid UI development
• Shared UI package - shadcn/ui primitives live in packages/ui
• tRPC - End-to-end type-safe APIs
• Bun - Runtime environment
• Drizzle - TypeScript-first ORM
• SQLite - Database engine
• Oxlint - Oxlint + Oxfmt (linting & formatting)

Getting Started

First, install the dependencies:

pnpm install

Database Setup

This project uses SQLite with Drizzle ORM.

1. Start the local SQLite database (optional):

pnpm run db:local

1. Update your .env file in the apps/server directory with the appropriate connection details if needed.

2. Apply the schema to your database:

pnpm run db:push

Then, run the development server:

pnpm run dev

Open http://localhost:3001 in your browser to see the web application. The API is running at http://localhost:3000.

Deployment

Docker Compose

• Target: web + server
• Config: docker-compose.yml (app Dockerfiles live in apps/*/Dockerfile)
• Build images: pnpm run docker:build
• Start: pnpm run docker:up
• Logs: pnpm run docker:logs
• Stop: pnpm run docker:down

Environment variables are read from each app's .env file (baked into web builds for public variables) and overridden in docker-compose.yml for container networking.

Docker Compose persists SQLite at ./.data/buncrawl.db and artifacts under ./.data/artifacts. The server applies its required storage schema during startup.

Project Structure

buncrawl/
├── apps/
│   ├── web/         # Frontend application (React + TanStack Router)
│   └── server/      # Backend API (Bun, TRPC)
├── packages/
│   ├── ui/          # Shared shadcn/ui components and styles
│   ├── api/         # API layer / business logic
│   └── db/          # Database schema & queries

Available Scripts

• pnpm run dev: Start all applications in development mode
• pnpm run build: Build all applications
• pnpm run dev:web: Start only the web application
• pnpm run dev:server: Start only the server
• pnpm run check-types: Check TypeScript types across all apps
• pnpm run db:push: Push schema changes to database
• pnpm run db:generate: Generate database client/types
• pnpm run db:migrate: Run database migrations
• pnpm run db:studio: Open database studio UI
• pnpm run db:local: Start the local SQLite database
• pnpm run check: Run Oxlint and Oxfmt
• pnpm run docker:build: Build the Docker Compose images
• pnpm run docker:up: Build and start the Docker Compose stack
• pnpm run docker:logs: Tail logs from the Docker Compose stack
• pnpm run docker:down: Stop the Docker Compose stack