1# Cloudflare AI Gateway
2 
3Expert guidance for implementing Cloudflare AI Gateway - a universal gateway for AI model providers with analytics, caching, rate limiting, and routing capabilities.
4 
5## When to Use This Reference
6 
7- Setting up AI Gateway for any AI provider (OpenAI, Anthropic, Workers AI, etc.)
8- Implementing caching, rate limiting, or request retry/fallback
9- Configuring dynamic routing with A/B testing or model fallbacks
10- Managing provider API keys securely with BYOK
11- Adding security features (guardrails, DLP)
12- Setting up observability with logging and custom metadata
13- Debugging AI Gateway requests or optimizing configurations
14 
15## Quick Start
16 
17**What's your setup?**
18 
19- **Using Vercel AI SDK** → Pattern 1 (recommended) - see [sdk-integration.md](./sdk-integration.md)
20- **Using OpenAI SDK** → Pattern 2 - see [sdk-integration.md](./sdk-integration.md)
21- **Cloudflare Worker + Workers AI** → Pattern 3 - see [sdk-integration.md](./sdk-integration.md)
22- **Direct HTTP (any language)** → Pattern 4 - see [configuration.md](./configuration.md)
23- **Framework (LangChain, etc.)** → See [sdk-integration.md](./sdk-integration.md)
24 
25## Pattern 1: Vercel AI SDK (Recommended)
26 
27Most modern pattern using official `ai-gateway-provider` package with automatic fallbacks.
28 
29```typescript
30import { createAiGateway } from 'ai-gateway-provider';
31import { createOpenAI } from '@ai-sdk/openai';
32import { generateText } from 'ai';
33 
34const gateway = createAiGateway({
35  accountId: process.env.CF_ACCOUNT_ID,
36  gateway: process.env.CF_GATEWAY_ID,
37});
38 
39const openai = createOpenAI({ 
40  apiKey: process.env.OPENAI_API_KEY 
41});
42 
43// Single model
44const { text } = await generateText({
45  model: gateway(openai('gpt-4o')),
46  prompt: 'Hello'
47});
48 
49// Automatic fallback array
50const { text } = await generateText({
51  model: gateway([
52    openai('gpt-4o'),              // Try first
53    anthropic('claude-sonnet-4-5'), // Fallback
54  ]),
55  prompt: 'Hello'
56});
57```
58 
59**Install:** `npm install ai-gateway-provider ai @ai-sdk/openai @ai-sdk/anthropic`
60 
61## Pattern 2: OpenAI SDK
62 
63Drop-in replacement for OpenAI API with multi-provider support.
64 
65```typescript
66import OpenAI from 'openai';
67 
68const client = new OpenAI({
69  apiKey: process.env.OPENAI_API_KEY,
70  baseURL: `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/compat`,
71  defaultHeaders: {
72    'cf-aig-authorization': `Bearer ${cfToken}` // For authenticated gateways
73  }
74});
75 
76// Switch providers by changing model format: {provider}/{model}
77const response = await client.chat.completions.create({
78  model: 'openai/gpt-4o', // or 'anthropic/claude-sonnet-4-5'
79  messages: [{ role: 'user', content: 'Hello!' }]
80});
81```
82 
83## Pattern 3: Workers AI Binding
84 
85For Cloudflare Workers using Workers AI.
86 
87```typescript
88export default {
89  async fetch(request, env, ctx) {
90    const response = await env.AI.run(
91      '@cf/meta/llama-3-8b-instruct',
92      { messages: [{ role: 'user', content: 'Hello!' }] },
93      { 
94        gateway: { 
95          id: 'my-gateway',
96          metadata: { userId: '123', team: 'engineering' }
97        } 
98      }
99    );
100    
101    return Response.json(response);
102  }
103};
104```
105 
106## Headers Quick Reference
107 
108| Header | Purpose | Example | Notes |
109|--------|---------|---------|-------|
110| `cf-aig-authorization` | Gateway auth | `Bearer {token}` | Required for authenticated gateways |
111| `cf-aig-metadata` | Tracking | `{"userId":"x"}` | Max 5 entries, flat structure |
112| `cf-aig-cache-ttl` | Cache duration | `3600` | Seconds, min 60, max 2592000 (30 days) |
113| `cf-aig-skip-cache` | Bypass cache | `true` | - |
114| `cf-aig-cache-key` | Custom cache key | `my-key` | Must be unique per response |
115| `cf-aig-collect-log` | Skip logging | `false` | Default: true |
116| `cf-aig-cache-status` | Cache hit/miss | Response only | `HIT` or `MISS` |
117 
118## In This Reference
119 
120| File | Purpose |
121|------|---------|
122| [sdk-integration.md](./sdk-integration.md) | Vercel AI SDK, OpenAI SDK, Workers binding patterns |
123| [configuration.md](./configuration.md) | Dashboard setup, wrangler, API tokens |
124| [features.md](./features.md) | Caching, rate limits, guardrails, DLP, BYOK, unified billing |
125| [dynamic-routing.md](./dynamic-routing.md) | Fallbacks, A/B testing, conditional routing |
126| [troubleshooting.md](./troubleshooting.md) | Debugging, errors, observability, gotchas |
127 
128## Reading Order
129 
130| Task | Files |
131|------|-------|
132| First-time setup | README + [configuration.md](./configuration.md) |
133| SDK integration | README + [sdk-integration.md](./sdk-integration.md) |
134| Enable caching | README + [features.md](./features.md) |
135| Setup fallbacks | README + [dynamic-routing.md](./dynamic-routing.md) |
136| Debug errors | README + [troubleshooting.md](./troubleshooting.md) |
137 
138## Architecture
139 
140AI Gateway acts as a proxy between your application and AI providers:
141 
142```
143Your App → AI Gateway → AI Provider (OpenAI, Anthropic, etc.)
144         ↓
145    Analytics, Caching, Rate Limiting, Logging
146```
147 
148**Key URL patterns:**
149- Unified API (OpenAI-compatible): `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions`
150- Provider-specific: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}/{endpoint}`
151- Dynamic routes: Use route name instead of model: `dynamic/{route-name}`
152 
153## Gateway Types
154 
1551. **Unauthenticated Gateway**: Open access (not recommended for production)
1562. **Authenticated Gateway**: Requires `cf-aig-authorization` header with Cloudflare API token (recommended)
157 
158## Provider Authentication Options
159 
1601. **Unified Billing**: Use AI Gateway billing to pay for inference (keyless mode - no provider API key needed)
1612. **BYOK (Store Keys)**: Store provider API keys in Cloudflare dashboard
1623. **Request Headers**: Include provider API key in each request
163 
164## Related Skills
165 
166- [Workers AI](../workers-ai/README.md) - For `env.AI.run()` details
167- [Agents SDK](../agents-sdk/README.md) - For stateful AI patterns
168- [Vectorize](../vectorize/README.md) - For RAG patterns with embeddings
169 
170## Resources
171 
172- [Official Docs](https://developers.cloudflare.com/ai-gateway/)
173- [API Reference](https://developers.cloudflare.com/api/resources/ai_gateway/)
174- [Provider Guides](https://developers.cloudflare.com/ai-gateway/usage/providers/)
175- [Discord Community](https://discord.cloudflare.com)
176