TokenShield：面向 LLM 与视觉 API 的多模态预推理优化网关

Multi-Modal Pre-Inference Optimization Gateway for LLMs & Vision APIs
Slash LLM API costs by 70% – 88% before sending a single token over the wire.

⚡ What is TokenShield?

TokenShield is a lightweight, local pre-inference optimization engine designed to dismantle the largest bottleneck in artificial intelligence: Token Consumption & API Overbilling.

Current AI proxies (LiteLLM, Portkey, Helicone) simply route raw requests directly to model providers. TokenShield acts as an intelligent optimization firewall — stripping out redundant boilerplate, intelligently cropping oversized high-res imagery, and matching perceptual visual hashes locally in milliseconds to achieve zero-token responses.

🏗️ Architecture

                       [USER / CLIENT REQUEST]
                     (Prompt + High-Res Image)
                                 │
                                 ▼
   ┌───────────────────────────────────────────────────────────┐
   │                  TOKENSHIELD GATEWAY                      │
   │                                                           │
   │   [ENGINE 1: Dynamic Context Aliasing (Text - CADVS)]     │
   │   Repeated Entities ──► Compact Registry ($V1, $V2)       │
   │                                                           │
   │   [ENGINE 3: Perceptual Visual Hash Cache (PV-Cache)]     │
   │   64-bit dHash ──► Hamming Match (≤4) ──► 0 TOKENS HIT!   │
   │                         │ (Cache Miss)                    │
   │                         ▼                                 │
   │   [ENGINE 2: Saliency-Based ROI Cropper (ASVPP)]          │
   │   1080p Image ──► Strip Blank Margins ──► 512x512 ROI     │
   └─────────────────────────────┬─────────────────────────────┘
                                 │
                                 ▼ (70% - 88% Fewer Tokens!)
                  [LLM API: Claude / GPT-4o / Gemini]

🚀 The 3 Core Engines

1. Engine 1: Context Aliasing & Dynamic Variable Substitution (CADVS)

• Problem: Byte-Pair Encoding (BPE) re-tokenizes long entity names, database schemas, and microservice tags repeatedly on every mention.
• Mechanism: Discovers candidate substring frequencies, extracts them into a compact single-token header dictionary ($V1, $V2), and replaces all body occurrences in-place.
• Guarantee: Lossless. Zero semantic degradation, unlike lossy token pruners that drop code punctuation.

2. Engine 2: Saliency-Based Region of Interest (ROI) Cropper (ASVPP)

• Problem: Vision models (GPT-4o, Claude 3.5 Sonnet) calculate tokens in 512x512 grid tiles (170 tokens/tile + 85 base). A 1080p or 4K screenshot incurs 1,500 – 2,500 tokens, where 80% is empty whitespace or inactive window borders.
• Mechanism: Evaluates grayscale contrast variances, extracts the information-dense content bounding box with a 10% safety buffer, and discards empty background canvas.
• Benchmark: Drops a 1080p terminal error screenshot from 2,125 tokens down to 595 tokens (72.0% – 88.0% direct token cut).

3. Engine 3: Perceptual Visual Hash Cache (PV-Cache)

• Problem: Conventional cryptographic hashes (MD5, SHA-256) fail if an image is slightly compressed, resized, or noisy, triggering redundant $0.02 API calls.
• Mechanism: Computes a 64-bit Difference Hash (dHash) on CPU in ~2ms. Matches against a local persistent SQLite registry using Hamming Distance tolerance ($\le 4$ bits).
• Benchmark: Detects heavily compressed JPEGs (Quality 40%) and resized mobile uploads with 93.8% – 100% similarity, returning verified responses with 0 TOKENS and 6.3ms latency.

📊 Verified Real-World Benchmark Scorecard

Tested against real PostgreSQL enterprise migration scripts and live 1080p Full HD terminal screenshots:

📦 Quickstart & Installation

Requirements

• Python 3.10+
• Pillow (pip install Pillow)

git clone https://github.com/harshitpratap1113-glitch/tokenshield.git
cd tokenshield

Run Live Benchmarks

# Run verified benchmark with real enterprise database queries and 1080p screenshots
python real_world_benchmark.py

Python API Usage

from gateway import TokenShieldGateway

# Initialize Gateway
gateway = TokenShieldGateway()

# 1. Optimize Text Prompt
optimized_text = gateway.optimize_text("Your long repetitive database prompt...")
print(optimized_text["compressed_text"])
print(f"Saved: {optimized_text['savings_pct']}% tokens")

# 2. Optimize Vision Input (Cache Check -> Saliency Crop)
optimized_vision = gateway.optimize_vision("path/to/screenshot_1080p.png")
if optimized_vision.get("source") == "VISUAL_CACHE_HIT":
    print("⚡ ZERO TOKENS! Cached response:", optimized_vision["cached_response"])
else:
    print(f"Cropped to ROI! Saved: {optimized_vision['savings_pct']}% vision tokens")

🔌 Model Context Protocol (MCP) Integration

TokenShield includes a native FastMCP Server (mcp_server.py) ready for 1-click integration with Claude Desktop, Cursor, Antigravity, and any MCP client!

Claude Desktop Configuration (claude_desktop_config.json):

{
  "mcpServers": {
    "tokenshield": {
      "command": "python",
      "args": ["D:\\TokenShield\\mcp_server.py"]
    }
  }
}

Exposed MCP Tools:

1. optimize_prompt_tokens: Compresses text prompts using Lossless Context Aliasing (CADVS).
2. optimize_vision_image: Scans images for 0-token cache hit or extracts tight ROI bounding box.
3. store_visual_cache: Saves verified image answers for future instant 0-token matching.
4. get_tokenshield_status: Checks engine health and lifetime cache entries.

🗺️ Roadmap

• Phase 1: Core Optimization Engines (CADVS, ASVPP, PV-Cache) & Real Benchmark Verification.
• Phase 2: Model Context Protocol (MCP) Server Wrapper (mcp_server.py) for Claude Desktop & Cursor.
• Phase 3: FastAPI Drop-in Reverse Proxy (http://localhost:8765/v1/chat/completions) for OpenAI SDK compatibility.
• Phase 4: Indic Byte Tokenizer (Devanagari byte compression saving 60% on Hindi prompts).

📜 License

MIT License. Created and maintained by harshitpratap1113-glitch. Contributions welcome!