API Proxy Architecture
ContextMiner is a drop-in reverse proxy. By simply changing your provider base URL from api.openai.com/v1 to https://context-miner.com/v1, your prompt passes through a local pipeline that scrubs sensitive PII, prunes structural syntax deadweight via our Safe Syntax Parser, and forwards the clean payload directly to your AI model.
Quick Start
Begin optimization by redirecting your requests. Select your programming environment below to view dependency setup and proxy code.
curl -X POST https://context-miner.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer cm_key_YOUR_API_KEY" \
-d '{
"model": "gpt-4o",
"messages": [
{"role": "user", "content": "Long HTML page or document here..."}
],
"stream": false
}'API Reference
Interact with the proxy using standard parameters.
Proxy completions endpoint that intercepts payloads, strips data, and forwards optimized strings downstream.
• Base URL vs Endpoint URL
If you are building custom API scripts (Python, cURL, Node), use the full endpoint URL: https://context-miner.com/v1/chat/completions. However, if you are configuring an IDE like Cursor, Cline, or Roo (or libraries like LangChain), they automatically append the chat path. For these apps, set your Base URL exactly to: https://context-miner.com/v1 (without the trailing slash).
API Headers
| Header Key | Default Value | Purpose |
|---|---|---|
| Content-Type | application/json | Specifies input payload structure. |
| Authorization | Bearer cm_key_YOUR_API_KEY | Your ContextMiner proxy API key. |
Request Body Parameters
| Field | Type | Requirement | Description |
|---|---|---|---|
| model | string | Required | Upstream model identifier (e.g. gpt-4o, deepseek-chat). |
| messages | array | Required | List of chat message objects containing role and content. |
| stream | boolean | Optional | Fully supported. Set to true to receive Server-Sent Events (SSE). The proxy translates Anthropic and Gemini stream formats to the OpenAI choices[].delta.content standard in real time. |
• Media Block Policy
To ensure absolute data sovereignty, low latencies, and prevent unmaskable visual PII leakage, ContextMiner blocks incoming media formats including images, audio, and video arrays when running in General Mode. Sending messages containing image_url, audio, or video properties in this mode immediately returns an HTTP 415 Unsupported Media Type status code. In IDE/Dev Mode, media and multimodal files are fully allowed.
Response Schema
The proxy returns standard OpenAI structures. The `prompt_tokens` in usage reports the final optimized token counts, which yields actual billing reductions downstream.
{
"id": "chatcmpl-123",
"object": "chat.completion",
"created": 1677652288,
"model": "gpt-4o",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Optimized output response from your AI provider."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 1260, // Post-compression prompt token count
"completion_tokens": 100, // Response token count
"total_tokens": 1360
}
}HTTP Error Codes & Resolution
| Status | Name | Primary Cause | Resolution |
|---|---|---|---|
| 400 | Bad Request | Malformed JSON payload or incorrect fields. | Check your parameters and ensure body is correct JSON. |
| 401 | Unauthorized | Missing, empty, or invalid Bearer key. | Validate your proxy API key (`cm_key_...`). |
| 402 | Payment Required | Subscription past due or unpaid invoice on Stripe. | Update your payment method in the Billing tab to restore API access immediately. |
| 409 | Key Conflict / Key Already in Use | The upstream AI provider key is already connected to another Free-tier account. | Upgrade to Pro to share this key across multiple accounts, or connect a unique API key. |
| 415 | Unsupported Media | Payload contains images, audio, or video while running in General Mode. | Remove media blocks or switch your connection to IDE/Dev Mode in the dashboard. |
| 429 | Quota Exceeded / Rate Limited | Monthly request limit reached (Free: 1,000, Pro: 100k, Enterprise: 500k). | Upgrade plan, add on-demand request packs, or wait for monthly cycle renewal. |
| 508 | Loop Detected | Recursive proxy call caught by the anti-loop header (x-contextminer-proxy). | Ensure your upstream connection does not point back to ContextMiner recursively. |
| 502 / 504 | Upstream Provider Error | Downstream model API (OpenAI, Anthropic, Gemini) timed out or returned 5xx. | Check your provider account status or retry with exponential backoff. |
Content Types
ContextMiner identifies document types dynamically to strip formatting and markup without compromising structural integrity.
Prunes CSS styles, inline javascript scripts, structural header blocks, and footer wrappers. Keeps text structured via Semantic Skeleton Extraction.
Prunes null entries, duplicate maps, blank configuration entries, and trailing layout spacing.
Prunes consecutive empty lines, margins, duplicate headers, and guide rows.
Deduplicates verbose repeating stack loops and standardizes common log output shapes.
Processes programs through our Structural Logic Analyzer to discard comments, empty statements, and formatting.
Normalizes header tags, collapses nested spacings, and strips decoration lines.
Structural Logic Analysis Comparison
Unlike raw text manipulation which can break syntax, the Structural Logic Analyzer understands code blocks. Comments, multi-line gaps, docstrings, and formatting margins are safely stripped. Logic remains fully compilable and semantically identical to the model.
/**
* Custom hook to track intersection
* Filters visible items on page
*/
export function useTracker(elementId) {
const [visible, setVisible] = useState(false);
const observerRef = useRef(null);
useEffect(() => {
// Check intersection support
if (typeof window !== 'undefined') {
observerRef.current = new IntersectionObserver(([e]) => {
setVisible(e.isIntersecting);
if (e.isIntersecting) {
console.log("Element visible: " + elementId);
}
});
const el = document.getElementById(elementId);
if (el) observerRef.current.observe(el);
}
return () => observerRef.current?.disconnect();
}, [elementId]);
return visible;
}export function useTracker(elementId){const[visible,setVisible]=useState(false);const observerRef=useRef(null);useEffect(()=>{if(typeof window!=='undefined'){observerRef.current=new IntersectionObserver(([e])=>{setVisible(e.isIntersecting)});const el=document.getElementById(elementId);if(el)observerRef.current.observe(el)}return()=>observerRef.current?.disconnect()},[elementId]);return visible}Engine Deep Dive
Behind the scenes, ContextMiner utilizes a dual-engine architecture to optimize prompts and balance data fidelity.
Structural Logic Analyzer & Deterministic Payload Minifier
Powered by the Dynamic Context Optimizer, the Structural Logic Analyzer loads programming syntax natively to strip comments, whitespaces, and unused libraries. Simultaneously, the Deterministic Payload Minifier specifically targets and collapses giant JSON payloads and structured trees. Upstream coding bots (e.g. Cursor) send highly condensed snippets without syntax breakage.
Reversible Context Compression (CCR) & Dynamic Context Optimizer
CCR identifies structural patterns and prompt prefix sequences in an in-memory cache. In IDE & Dev Mode (40–55%+ savings, 45% floor), it preserves code structure, function schemas, and local test endpoints. In General Mode (60–80%+ savings, 20% floor), dynamic Semantic Relevance Filtering and Web Noise Filtering compress large prompts without losing conversational coherence.
Selective Multimodal Filtering & Zero-Latency Gateway
In General Mode, non-text media files are filtered to maximize text token savings. In IDE & Dev Mode, all images (UI screenshots, mockups), PDFs, documents, and multimodal files accepted by upstream AIs are fully allowed and routed downstream.
Sub-millisecond Offline Tokenizer Heuristic
Evaluates input token volume locally in microseconds (<0.1ms) using model-calibrated heuristics, eliminating upstream count queries and network roundtrips.
Explore Offline Tokenizer & Cost Calibration Multipliers→Estimated Savings Formulation
Savings are logged asynchronously. Input prompt tokens are priced based on the upstream cost definitions. The formulation is:
Offline Tokenizer & Cost Calibration
Sub-millisecond Lexical BPE ApproximationContextMiner uses a lexical BPE approximation formula to estimate prompt token usage offline before sending it upstream. It matches official token counts with 98% accuracy by mapping operators, symbols, whitespaces, and long words.
Model-Specific Multipliers
Calibrated multipliers applied to normalize BPE estimates against proprietary upstream tokenizers.
| Provider Families | Token Multiplier | Heuristic Reason |
|---|---|---|
| OpenAI (GPT-4o, o1, o3) | 0.90x | Calibrated for the efficient o200k_base vocabulary. |
| Anthropic (Claude 3.5 Sonnet) | 0.95x | Slightly denser token mapping than generic cl100k. |
| DeepSeek (V3, R1) | 0.92x | DeepSeek vocabulary optimizations. |
| Google Gemini (1.5 Pro/Flash) | 1.05x | SentencePiece layout structures, slightly less dense on code blocks. |
Real-time ROI & Savings Computation
Savings are computed instantly and recorded asynchronously without holding up streaming response chunks. Prompt tokens saved are converted directly into dollar amounts based on target provider model pricing:
Compliance Shield
Enterprise PII anonymization. Replaces sensitive customer data with standardized placeholders before it leaves your network.
Zero-Egress Security Model
All masking runs locally on CPU via fast NLP entity recognition and deterministic pattern detection. Thanks to our Token Vault technology, original data is fully reconstructed for your end-users.
Supported Global Frameworks
Compliance Framework Mapping
LGPD (Brazil)
General Data Protection Law (LGPD)
Protects the personal information of Brazilian citizens. Masking automatically matches formatted and unformatted versions of CPF, CNPJ, RG, CNH, PIS, Voter Registration Card, credit cards, emails, and Brazilian phone numbers to align with data minimization principles.
HIPAA (USA)
Health Insurance Portability & Accountability Act
Protects health data (PHI) in the United States. Under Safe Harbor regulations, direct identifiers must be de-identified. ContextMiner masks Social Security Numbers (SSN), Medicare Beneficiary Identifiers (MBI), emails, and phone numbers.
CCPA (California)
California Consumer Privacy Act
Grants privacy rights to California consumers. Automatically anonymizes network IDs like IP and MAC Addresses, personal identifiers such as SSNs, emails, and phone numbers to prevent downstream data mining.
GDPR (Europe)
General Data Protection Regulation
Enforces data security for EU citizens. Automatically redacts bank coordinates like IBAN, credentials including Passport Numbers, emails, IP addresses, and MAC addresses.
UK DPA 2018
Data Protection Act 2018
UK implementation of GDPR principles. Identifies and redacts British identity document layouts including National Insurance Numbers (NIN), UK Driving Licenses, UK Passport formats, bank coordinates (IBAN), and email directories.
Masking Replacements Directory
Personal identifiers are replaced by Reversible Tokens (e.g. `{{TKN_1A2B}}`). Whether passed formatted (with dots, dashes, spaces, or slashes) or completely unformatted (e.g. raw digit strings), the shield identifies them and restores them upon response.
LGPD / Brazilian Identification Forms
123.456.789-001234567890012.345.678/0001-001234567800010012.345.678-9RG: 123456789CNH: 12345678901Título: 1234 567 890 12Título: 123456789012120.55555.21-9PIS: 12055555219HIPAA & CCPA / US Identification Forms
123-45-67891234567891EG4-TE5-MK721EG4TE5MK72GDPR & UK DPA / European Identifiers
DE89 3704 0044 0532 0130 00DE89370400440532013000QQ123456AJONES123456AB9CDPassport: 123456789C12345678General Contact & Network Coordinates
user@domain.com+55 11 99999-999911999999999+1 (555) 123-456755512345674111 2222 3333 4444192.168.1.1ip: 1921681100:1A:2B:3C:4D:5ECredentials & Secrets (Critical Security)
eyJhbGciOi...AKIAIOSFODNN7...-----BEGIN RSA PRIVATE KEY-----sk-proj-1234567...• Prompt Injection Guardrails & NLP
Beyond deterministic boundary safety, ContextMiner integrates a lightweight NLP NER engine to identify complex, unstructured entities (names, cities, organizations). It also implements Prompt Injection Guardrails to intercept Jailbreak attacks (e.g. "ignore all instructions") at the proxy level before they consume your API tokens.
Universal Custom OpenAI API Connector
Bring Your Own Key. ContextMiner standardizes on the Universal Custom OpenAI API Connector (OpenAI-Compatible protocol) to route downstream calls to any LLM provider or self-hosted engine with zero code changes.
OpenAI & OpenRouter
gpt-4o, gpt-4o-mini, o1, o3-mini, all OpenRouter models
DeepSeek & Groq
deepseek-chat, deepseek-reasoner, llama-3.3-70b-versatile
Gemini & Self-Hosted
gemini-1.5-pro (OpenAI endpoint), Ollama, vLLM, LM Studio
Universal OpenAI-Compatible Connection Specifications
| Connection Standard | Target Provider Base URL Example | Authentication Header Key | API Schema Format |
|---|---|---|---|
| Custom OpenAI (OpenAI Official) | https://api.openai.com/v1 | Authorization: Bearer [KEY] | OpenAI Compatible |
| Custom OpenAI (OpenRouter) | https://openrouter.ai/api/v1 | Authorization: Bearer [KEY] | OpenAI Compatible |
| Custom OpenAI (DeepSeek) | https://api.deepseek.com | Authorization: Bearer [KEY] | OpenAI Compatible |
| Custom OpenAI (Groq) | https://api.groq.com/openai/v1 | Authorization: Bearer [KEY] | OpenAI Compatible |
| Custom OpenAI (Google Gemini) | https://generativelanguage.googleapis.com/v1beta/openai | Authorization: Bearer [KEY] | OpenAI Compatible |
| Custom OpenAI (Ollama / vLLM) | http://localhost:11434/v1 or http://localhost:8000/v1 | Authorization: Bearer [KEY] | OpenAI Compatible |
Agent & IDE Integrations
Forward coding assistant traffic through the local proxy.
1. Cursor IDE Setup
Cursor allows configuring custom OpenAI-compatible models. By setting the endpoint base to the ContextMiner proxy, it routes coding queries with ultra-low latency.
cm_key_YOUR_API_KEY.https://context-miner.com/v1.gpt-4o) and test the chat.2. Aider CLI Setup
Configure Aider to route prompt and codebase contexts through ContextMiner using environment variables.
.env file:~/.aider.conf.yml.3. Cline / Roo Code Assistants
VS Code extensions (Cline, Roo Code) supporting Custom OpenAI endpoints can be configured similarly.
https://context-miner.com/v1.cm_key_YOUR_API_KEY.Fair Use & Anti-Abuse Policy
View Pricing & Plans →To protect platform uptime and ensure high-speed processing for all developers and production systems, ContextMiner enforces transparent fair use guardrails and anti-abuse policies.
Fair Use & Anti-Abuse Guardrails
To protect our servers from scripted loops, preserve low latency, and keep the service reliable, we enforce four core guardrails:
We allow 1 active free account per IP network. If you need multiple developers or agents on the same company network, upgrade to Pro or reach out to our team for custom provisioning.
Each upstream AI provider key (OpenAI, Anthropic, Gemini) can be connected to one Free account at a time. We calculate an irreversible SHA-256 hash of your key upon saving—your raw secrets are never logged or exposed in plain text. Pro and Enterprise accounts have no key reuse limits.
Temporary or disposable email domains (such as 10minutemail or tempmail) are blocked during registration. We normalize email aliases so accounts stay clean and genuine.
We record strictly numerical token counts and byte metrics in our database. Your code snippets, system prompts, chat completions, and API keys are never written to disk or used for model training.
