Streaming Completions
Stream real-time tokens over Server-Sent Events (SSE) using Zorveus low-latency proxy.
Streaming Completions
POST/v1/chat/completions
Inference Key (Bearer zrv_...)
Setting "stream": true streams tokens incrementally as Server-Sent Events (SSE) chunks, allowing your UI to display characters as they are generated.
Inference Key TesterInteractive
Paste your Zorveus key to dynamically update all code snippets below.
Code Examples
from openai import OpenAI
client = OpenAI(
base_url="https://api.zorveus.com/v1",
api_key="zrv_your_inference_key",
)
stream = client.chat.completions.create(
model="openai/gpt-4.1-mini",
messages=[
{"role": "user", "content": "Write a haiku about distributed systems."}
],
stream=True,
)
for chunk in stream:
content = chunk.choices[0].delta.content or ""
print(content, end="", flush=True)
print()Usage Accounting for Streams
When streaming, token usage is accumulated by the Zorveus Gateway on the fly. The final usage event is settled immediately upon stream termination ([DONE] event) or client disconnection.
Was this page helpful?
Edit this page on GitHub