Developer Docs

Connect to a production-grade AI gateway in 5 minutes. A complete guide to unified endpoints, failover, semantic caching and PII masking.

Quickstart

coolbestllm is fully compatible with the OpenAI API. Change just one line, your Base URL, to add routing, failover, caching and masking.

quickstart.sh

export OPENAI_BASE_URL="https://api.coolbestllm.com/v1"
export COOLBEST_KEY="sk-cbl-..."

curl $OPENAI_BASE_URL/chat/completions \
  -H "Authorization: Bearer $COOLBEST_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Hi"}]}'

Authentication

All requests authenticate through Authorization: Bearer <KEY>. When creating a Key in the console, you can assign a model allowlist and a Token budget cap.

Unified Endpoints

Endpoint	Description
`/v1/chat/completions`	Chat completions (OpenAI-compatible)
`/v1/embeddings`	Vector embeddings
`/v1/models`	List available models

Failover

Configure an ordered list of backup instances for a model. If the primary channel fails, the gateway automatically falls back within 100ms.

routing.json

{
  "model": "gpt-4o",
  "fallbacks": ["azure/gpt-4o", "claude-3.5-sonnet"],
  "timeout_ms": 100,
  "retry": 2
}

Semantic Caching

Enable semantic caching through request headers to serve similar questions directly from the cache and save 30–50% in Tokens.

headers

X-CBL-Cache: semantic
X-CBL-Cache-TTL: 3600
X-CBL-Cache-Threshold: 0.92

PII Masking

When enabled, the gateway automatically identifies and masks sensitive fields, such as names, email addresses, ID numbers and card numbers, before forwarding requests upstream.

headers

X-CBL-PII-Masking: on
X-CBL-PII-Entities: name,email,phone,card,id
X-CBL-Data-Retention: zero

SDK Examples

node.js

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.coolbestllm.com/v1",
  apiKey: process.env.COOLBEST_KEY,
});

const r = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello" }],
});