Vercel AI SDK

Add observability, reliability, and cost control to Vercel AI apps.

Portkey integrates natively with the Vercel AI SDK to add production features:

• Full observability (costs, latency, logs)
• Access to 250+ LLMs through one interface
• Automatic fallbacks, retries, and caching
• Budget controls and guardrails

Quick Start

import { createPortkey } from '@portkey-ai/vercel-provider';
import { generateText } from 'ai';

const portkey = createPortkey({
  apiKey: 'PORTKEY_API_KEY'
});

const { text } = await generateText({
  model: portkey.chatModel('@openai-prod/gpt-4o'),
  prompt: 'What is Portkey?'
});

console.log(text);

All requests now route through Portkey with full observability.

Setup

1. Install Package

npm install @portkey-ai/vercel-provider

2. Add Provider in Model Catalog

1. Go to Model Catalog → Add Provider
2. Select your provider (OpenAI, Anthropic, Google, etc.)
3. Enter your API keys
4. Name your provider (e.g., openai-prod)

Your provider slug is @openai-prod (or whatever you named it).

Model Catalog Guide →

Set up budgets, rate limits, and manage credentials

3. Get Portkey API Key

Create your API key at app.portkey.ai/api-keys.

Pro tip: Attach a default config to your API key for features like fallbacks and caching—no code changes needed.

4. Use in Code

import { createPortkey } from '@portkey-ai/vercel-provider';

const portkey = createPortkey({
  apiKey: 'PORTKEY_API_KEY'
});

// Use provider slug + model name
const response = await generateText({
  model: portkey.chatModel('@openai-prod/gpt-4o'),
  prompt: 'Hello!'
});

Vercel Functions

Works with all Vercel AI functions:

generateText

import { createPortkey } from '@portkey-ai/vercel-provider';
import { generateText } from 'ai';

const portkey = createPortkey({ apiKey: 'PORTKEY_API_KEY' });

const { text } = await generateText({
  model: portkey.chatModel('@openai-prod/gpt-4o'),
  prompt: 'What is Portkey?'
});

console.log(text);

Tool Calling

import { createPortkey } from '@portkey-ai/vercel-provider';
import { generateText, tool } from 'ai';
import { z } from 'zod';

const portkey = createPortkey({ apiKey: 'PORTKEY_API_KEY' });

const result = await generateText({
  model: portkey.chatModel('@openai-prod/gpt-4o'),
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for')
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10
      })
    })
  },
  prompt: 'What is the weather in San Francisco?'
});

Switching Providers

Change the model string to switch providers:

// OpenAI
portkey.chatModel('@openai-prod/gpt-4o')

// Anthropic
portkey.chatModel('@anthropic-prod/claude-sonnet-4')

// Google
portkey.chatModel('@google-prod/gemini-2.0-flash')

No other code changes needed.

Advanced Features

For fallbacks, caching, and load balancing, create a Config and attach it to your API key. The config applies automatically—no code changes required.

Fallbacks

Auto-switch to backup models if the primary fails:

{
  "strategy": { "mode": "fallback" },
  "targets": [
    { "override_params": { "model": "@openai-prod/gpt-4o" } },
    { "override_params": { "model": "@anthropic-prod/claude-sonnet-4" } }
  ]
}

Load Balancing

Distribute requests across providers:

{
  "strategy": { "mode": "loadbalance" },
	"targets": [
    { "override_params": { "model": "@openai-prod/gpt-4o" }, "weight": 0.7 },
    { "override_params": { "model": "@anthropic-prod/claude-sonnet-4" }, "weight": 0.3 }
	]
}

Caching

Reduce costs with response caching:

{
  "cache": { "mode": "semantic" },
  "override_params": { "model": "@openai-prod/gpt-4o" }
}

Runtime Config

Pass config inline when needed:

const portkey = createPortkey({
  apiKey: 'PORTKEY_API_KEY',
  config: {
    strategy: { mode: 'fallback' },
    targets: [
      { override_params: { model: '@openai-prod/gpt-4o' } },
      { override_params: { model: '@anthropic-prod/claude-sonnet-4' } }
    ]
  }
});

// Model comes from config, so pass empty string
const { text } = await generateText({
  model: portkey.chatModel(''),
  prompt: 'What is Portkey?'
});

Configs Guide →

Fallbacks, retries, caching, load balancing, and more

Guardrails

Add input/output validation via config:

		{
  "input_guardrails": ["guardrail-id-xxx"],
  "output_guardrails": ["guardrail-id-yyy"],
  "override_params": { "model": "@openai-prod/gpt-4o" }
}

Guardrails Guide →

PII detection, content filtering, and custom rules

Observability

All requests are automatically logged with:

• Cost and token usage
• Latency metrics
• Full request/response payloads
• Custom metadata

Observability Guide →

Track costs, performance, and debug issues

Next Steps

Model Catalog

Set up providers and budgets

Configs

Configure fallbacks and routing

Observability

Track costs and performance

Guardrails

Add PII detection and filtering