跳到正文
GitHub·· 9 天前AI 评分22

Nexora:面向团队的文档溯源 RAG 工作空间

elmira-orooji/Nexora-Advanced-RAG

AI 导读

Nexora 是一个面向团队的文档溯源 AI 工作空间,把已批准文档和连接的数据源整理成可检索、可讨论、可核验引用的知识集合。

正文

Nexora is a document-grounded AI workspace for teams. It turns approved documents and connected sources into knowledge collections that people can search, discuss, and verify through cited answers.

Caution

This is proprietary, source-available software. Public access does not grant permission to use, copy, modify, distribute, deploy, or contribute to Nexora. See LICENSE, CONTRIBUTING.md, and SECURITY.md.

Nexora product tour

The project follows a clear path from knowledge to answer:

Create a knowledge set → add documents or sources → process and index content → ask a question → inspect sources and answer confidence

What Nexora includes

  • Grounded conversations — retrieve relevant document chunks and present answer sources alongside a confidence state.
  • Knowledge sets — organize documents into focused collections with per-user access levels.
  • Document ingestion — upload PDF, TXT, JPEG, PNG, and TIFF files; track queue, extraction, OCR, chunking, and indexing progress.
  • Optional OCR — extract content from scanned documents with MinerU, Jina OCR, Google Vision, or Azure Document Intelligence.
  • Assistants — connect assistant behavior to one or more permitted knowledge sets.
  • Controlled sharing and feedback — publish revocable read-only conversation snapshots and collect answer feedback.
  • Connectors — synchronize supported website, GitHub, Google Drive, S3, SharePoint, and webhook sources through a background scheduler.
  • Team boundaries — separate organizations, roles, document-set permissions, and user sessions.
  • Bilingual interface — Persian and English UI, light and dark themes, and responsive layouts.

Architecture at a glance

Nexora uses a React/Vite frontend, a FastAPI backend, PostgreSQL for application data, and Qdrant for vector retrieval. The API, document worker, and connector scheduler are separate processes that share the same database, document storage, and Qdrant configuration.

Browser → React UI → FastAPI API → PostgreSQL
                              ↘ → Qdrant
                              ↘ → OpenRouter-compatible LLM

Document worker → storage → extraction/OCR → chunking and indexing → Qdrant
Connector scheduler → permitted remote sources → document processing queue

See ARCHITECTURE.md for component boundaries, data flow, worker behavior, and RAG retrieval details.

Quick start with Docker

Docker Compose is the recommended way to run the full local stack. It starts PostgreSQL, Qdrant, the API, document worker, connector scheduler, and frontend together.

Requirements

  • Docker Desktop or Docker Engine with Docker Compose v2
  • At least 4 GB of free memory for a comfortable local stack

Configure the environment

From the repository root:

Copy-Item .env.docker.example .env

Open .env and replace the database password in both POSTGRES_PASSWORD and DATABASE_URL. Also set a random AUTH_SECRET_KEY of at least 32 characters. Keep the included development cookie settings for local HTTP only.

Start the stack

docker compose up --build -d
docker compose ps

Open http://localhost:5173. The first start applies Alembic migrations before the API and workers begin.

To confirm that PostgreSQL, Qdrant, document storage, and both workers are ready:

Invoke-WebRequest http://localhost:5173/api/ready | Select-Object -ExpandProperty Content

For logs from a single service:

docker compose logs -f document-worker

The Compose setup limits the document worker to 0.75 CPU cores and 2 GB of memory by default. This limits local resource contention, but it can make indexing slower. Adjust those limits in compose.yml only after considering the capacity of the host.

Run without Docker

Use this option when PostgreSQL and Qdrant are already available elsewhere.

Backend

cd backend
py -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
Copy-Item .env.example .env

Set, at minimum, the following values in backend/.env:

DATABASE_URL=postgresql+psycopg2://USER:PASSWORD@HOST:5432/DATABASE
AUTH_SECRET_KEY=replace-with-a-random-value-of-at-least-32-characters
APP_ENV=development
AUTH_COOKIE_SECURE=false
FRONTEND_ORIGINS=http://localhost:5173,http://127.0.0.1:5173

QDRANT_URL=http://YOUR-QDRANT-HOST:6333
QDRANT_API_KEY=
QDRANT_COLLECTION=rag_chunks

OPENROUTER_API_KEY=
OPENROUTER_MODEL=openrouter/free

Apply migrations and start the three backend processes in separate terminals:

cd backend
.\.venv\Scripts\Activate.ps1
alembic upgrade head

# Terminal 1
uvicorn app.main:app --host 0.0.0.0 --port 8000

# Terminal 2
python -m app.workers.document_worker

# Terminal 3
python -m app.workers.connector_scheduler_worker

Frontend

cd frontend/my-rag-app
npm install
npm run dev

The development server runs on http://localhost:5173 and proxies /api calls to http://127.0.0.1:8000.

Document processing and OCR

PDF files with a text layer and TXT files are extracted locally. For scanned files, the default OCR route tries Jina first and falls back to MinerU if Jina is unavailable, fails, times out, or returns no text. Both are hosted services and receive document content; set OCR_PROVIDER=disabled to disable cloud OCR.

To enable MinerU for Persian scanned documents:

OCR_PROVIDER=mineru
MINERU_API_TOKEN=your-token
MINERU_LANGUAGE=fa
MINERU_MODEL_VERSION=vlm

Set OCR_PROVIDER=auto to try configured providers in this order: MinerU, Google Vision, and Azure Document Intelligence. Before enabling a cloud OCR provider, confirm that sending the original document to that provider is permitted by the organization’s data policy.

Jina OCR can be configured as the primary provider without installing a local model. MinerU is used as its fallback when configured:

OCR_PROVIDER=jina
JINA_API_KEY=your-jina-api-key
JINA_OCR_MODEL=jina-ocr-v1

With OCR_PROVIDER=jina, Jina is attempted first and MinerU second. Use this route only when the organization approves sending document content to both providers. If either credential is missing, the configured provider list adjusts accordingly.

The application does not currently include malware scanning. Do not treat MIME checks and file-size limits as an antivirus control. Before processing untrusted files in production, deploy a quarantine-and-scan workflow that blocks files when the scanner is unavailable. SECURITY.md defines the required controls.

Operations

Endpoint Purpose
GET /health Lightweight API liveness check.
GET /ready Readiness check for PostgreSQL, Qdrant, writable document storage, document worker, and connector scheduler.

/ready returns 503 until all required dependencies and worker heartbeats are available. This is intentional and makes it suitable for deployment readiness probes.

For production, use HTTPS, set APP_ENV=production, set AUTH_COOKIE_SECURE=true, restrict FRONTEND_ORIGINS to the real frontend origin, and keep PostgreSQL and Qdrant off the public network. Full deployment guidance is available in DEPLOYMENT.md.

Quality checks

The repository has separate backend and frontend checks:

# Backend
cd backend
.\.venv\Scripts\Activate.ps1
python -m pytest

# Frontend unit tests and production build
cd frontend/my-rag-app
npm run test
npm run build

# Frontend end-to-end tests
npm run test:e2e

GitHub Actions runs backend tests on pushes and pull requests to dev and main. A separate frontend workflow runs unit tests, a production build, and Playwright end-to-end tests on GitHub-hosted runners, so those checks do not consume local CPU while working on the project.

Repository layout

backend/                    FastAPI routes, models, migrations, workers, and tests
frontend/my-rag-app/        React application, component tests, and Playwright tests
documents/                  Product and requirements documents
compose.yml                 Full local Docker Compose stack
ARCHITECTURE.md             System design and data flows
DEPLOYMENT.md               Docker and production deployment guide
SECURITY.md                 Security posture and operational requirements
BACKUP_AND_RECOVERY.md      Backup and recovery procedures
DESIGN.md                   UI design system rules

Documentation

Troubleshooting

Symptom First checks
A document remains at 0% or 10% Confirm document-worker is running, then inspect /ready and the document processing error.
A scanned document fails Check OCR_PROVIDER, provider credentials, page limits, and OCR timeout settings.
The frontend reports ECONNREFUSED 127.0.0.1:8000 Start the FastAPI API on port 8000 or correct the development proxy target.
/ready returns 503 Check PostgreSQL, Qdrant, writable storage, document worker, and connector scheduler.
A connector does not synchronize Check the scheduler, connector configuration, lease state, and the last synchronization error.

Security note

Do not commit .env files, passwords, API keys, OAuth refresh tokens, Qdrant keys, OCR credentials, uploaded data, or backups. Review SECURITY.md before exposing the application to users or connecting it to organization data. Report vulnerabilities privately; do not use public GitHub issues for sensitive security details.

来源:GitHub · github.com