Streaming completions

Setting "stream": true streams tokens incrementally as Server-Sent Events (SSE) chunks, allowing your UI to display characters as they are generated.

Code examples

from openai import OpenAI

client = OpenAI(
    base_url="https://api.zorveus.com/v1",
    api_key="zrv_your_api_key",
)

stream = client.chat.completions.create(
    model="openai/gpt-4.1-mini",
    messages=[
        {"role": "user", "content": "Write a haiku about distributed systems."}
    ],
    stream=True,
)

for chunk in stream:
    content = chunk.choices[0].delta.content or ""
    print(content, end="", flush=True)

print()

Usage accounting for streams

Zorveus records the final provider outcome and available token usage after the stream completes or disconnects. A client disconnect does not guarantee that the upstream provider stopped before producing more billable output.

Preserve incomplete SSE lines between network chunks. Treat a stream as complete only after data: [DONE] or a documented terminal event. If you retry after a disconnect, the provider can bill both requests.