---
title: Inference overview
description: Overview of multi-provider wire protocols and inference capabilities supported on api.zorveus.com.
---

Zorveus sits between your applications and AI model providers, natively supporting the **Anthropic `/v1/messages`** wire protocol, standard **OpenAI `/v1/chat/completions`**, **Google Gemini**, and specialized audio, embedding, and realtime endpoints.

<KeyInserter type="api" label="API Key Tester" placeholder="zrv_your_api_key" />

---

## Supported wire protocols and endpoints

| Provider format | Endpoints | Example SDK usage |
| :--- | :--- | :--- |
| **Anthropic Native** | `POST /v1/messages`<br />`POST /v1/messages/count_tokens` | Standard Anthropic SDK (`@anthropic-ai/sdk`, `anthropic-python`) |
| **OpenAI Chat Format** | `POST /v1/chat/completions`<br />`POST /chat/completions` | Standard OpenAI SDK (`openai`), Vercel AI SDK, LangChain |
| **OpenAI Responses API** | `POST /v1/responses`<br />`POST /responses` | OpenAI Responses API format |
| **Google Gemini Native** | `POST /v1beta/models/*`<br />`POST /vertex_ai/live` | Google GenAI SDK and Vertex AI live stream |
| **Embeddings and Rerank** | `POST /v1/embeddings`<br />`POST /v1/rerank`, `POST /v2/rerank` | Cohere, Voyage, Jina, OpenAI embeddings and rerankers |
| **Audio and Speech** | `POST /v1/audio/speech`<br />`POST /v1/audio/transcriptions` | OpenAI Whisper, TTS, Gemini Audio |
| **Realtime Voice / WebSockets** | `POST /v1/realtime/calls`<br />`POST /v1/realtime/client_secrets`<br />`WS /v1/realtime` | OpenAI Realtime API, WebRTC, and WebSocket client sessions |

---

## Inference modalities

<CardGroup cols={2}>
  <Card title="Chat Completions" icon="terminal" href="/inference/chat-completions">
    Standard and reasoning completions across OpenAI, Anthropic, Gemini, and open models.
  </Card>
  <Card title="Streaming Completions" icon="zap" href="/inference/streaming">
    Server-Sent Events (SSE) token streaming.
  </Card>
  <Card title="Tool Calling & Function Calling" icon="code" href="/inference/tool-calling">
    Structured schema execution and agentic tool invocation.
  </Card>
  <Card title="Responses API" icon="layers" href="/inference/responses">
    OpenAI-compatible response items for supported provider routes.
  </Card>
  <Card title="Embeddings" icon="cpu" href="/inference/embeddings">
    Dense text embeddings for semantic search and retrieval (RAG).
  </Card>
  <Card title="Model Discovery" icon="key" href="/inference/model-discovery">
    Inspect dynamically available models and routing policy status.
  </Card>
</CardGroup>

---

## SDK configuration

Different SDK clients append different default path prefixes. Configure the exact base URL matching your SDK:

| SDK Client | Base URL parameter | Notes |
| :--- | :--- | :--- |
| **OpenAI Python / TypeScript** | `https://api.zorveus.com/v1` | OpenAI clients automatically append `/chat/completions`. |
| **Anthropic Python / TypeScript** | `https://api.zorveus.com` | Anthropic clients automatically append `/v1/messages`. Do not add `/v1`. |
| **Zorveus SDK (@zorveus/sdk)** | Default preconfigured | Connects automatically to `https://api.zorveus.com`. |
| **Direct HTTP / cURL** | Full route path | Use `https://api.zorveus.com/v1/chat/completions` or `https://api.zorveus.com/v1/messages`. |

<Callout type="warning" title="Avoid Anthropic base URL 404s">
  Do not append `/v1` to the base URL when using the official Anthropic SDK (`@anthropic-ai/sdk` or `anthropic-python`). The Anthropic client appends `/v1/messages` on its own. Setting `baseURL: "https://api.zorveus.com/v1"` results in an invalid `https://api.zorveus.com/v1/v1/messages` request.
</Callout>

Point the base URL to Zorveus in your client initialization:

<CodeGroup titles={["Anthropic (Python)", "Anthropic (TypeScript)", "OpenAI (Python)", "OpenAI (TypeScript)", "cURL"]}>

```python
import anthropic

client = anthropic.Anthropic(
    base_url="https://api.zorveus.com",
    api_key="zrv_your_api_key",
)

response = client.messages.create(
    model="claude-3-7-sonnet-20250219",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Explain quantum computing in one sentence."}
    ],
)
print(response.content[0].text)
```

```typescript
import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic({
  baseURL: "https://api.zorveus.com",
  apiKey: "zrv_your_api_key",
});

const message = await anthropic.messages.create({
  model: "claude-3-7-sonnet-20250219",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Explain quantum computing in one sentence." }],
});

console.log(message.content);
```

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.zorveus.com/v1",
    api_key="zrv_your_api_key",
)

response = client.chat.completions.create(
    model="openai/gpt-4.1-mini",
    messages=[{"role": "user", "content": "Explain quantum computing in one sentence."}],
)
print(response.choices[0].message.content)
```

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.zorveus.com/v1",
  apiKey: "zrv_your_api_key",
});

const completion = await client.chat.completions.create({
  model: "openai/gpt-4.1-mini",
  messages: [{ role: "user", content: "Explain quantum computing in one sentence." }],
});

console.log(completion.choices[0].message.content);
```

```bash
curl https://api.zorveus.com/v1/models \
  -H "Authorization: Bearer zrv_your_api_key"
```

</CodeGroup>

---

## How Zorveus handles requests under the hood

1. **Authentication and metering**: The gateway intercepts your request (`x-api-key: zrv_...` or `Authorization: Bearer zrv_...`), validates the API key, and verifies active balance and credit caps.
2. **BYOK injection and system routing**: If your organization configured custom provider credentials (e.g. Anthropic or OpenAI API keys), Zorveus securely injects them with automatic multi-key failover. Otherwise, requests route through Zorveus system credentials.
3. **Usage settlement**: The gateway calculates token consumption and cost, then settles the ledger event in real time upon completion.