Streaming

Server-sent events, keepalives and the usage chunk.

Set "stream": true to receive the response as server-sent events (SSE). Every OpenAI SDK handles this for you.

stream = client.chat.completions.create(
    model="deepseek/deepseek-v4-flash",
    messages=[{"role": "user", "content": "Write a haiku about routers."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:  # final chunk: one choice with an empty delta, usage present
        print("\ncost (USD):", chunk.usage.cost)

Wire format

• Each event is a line data: {chunk} followed by a blank line. Chunks are OpenAI chat.completion.chunk objects.
• While the model is still thinking, we send SSE comment lines (: keepalive) so proxies do not close the connection. Clients must ignore lines starting with :; SDKs already do.
• Reasoning models stream their trace first, in delta.reasoning_content and delta.reasoning (the same text in both, on every route). Reasoning is output: if you close the stream while the model is still reasoning, you pay for the reasoning already sent.
• The last data chunk carries a usage object including cost. It keeps one choice with an empty delta (repeating the finish_reason), so code that reads choices[0].delta on every chunk keeps working.
• The stream ends with data: [DONE].

: keepalive

data: {"id":"gen-…","object":"chat.completion.chunk","model":"deepseek/deepseek-v4-flash","choices":[{"index":0,"delta":{"role":"assistant","content":"Routers"}}]}

data: {"id":"gen-…","object":"chat.completion.chunk","model":"deepseek/deepseek-v4-flash","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"gen-…","object":"chat.completion.chunk","model":"deepseek/deepseek-v4-flash","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":14,"completion_tokens":17,"total_tokens":31,"cost":0.000009}}

data: [DONE]

Errors mid-stream

Once streaming has started the HTTP status is already 200, so a failure after that point arrives in-band. If the model provider reports an error or the connection to it drops, we send:

1. an error chunk: one choice with an empty delta and finish_reason: "error", plus an error object with type: "upstream_error", code: 503, retryable: true and charged (true when the output already sent was billed);
2. the usual final usage chunk, with the cost actually charged;
3. data: [DONE].

Check for an error field before treating a stream as complete. Once a stream has started it is not retried on another provider.

data: {"id":"gen-…","object":"chat.completion.chunk","model":"deepseek/deepseek-v4-flash","choices":[{"index":0,"delta":{},"finish_reason":"error"}],"error":{"message":"The upstream connection dropped mid-stream.","type":"upstream_error","code":503,"retryable":true,"charged":true}}

data: {"id":"gen-…","object":"chat.completion.chunk","model":"deepseek/deepseek-v4-flash","choices":[{"index":0,"delta":{},"finish_reason":null}],"usage":{"prompt_tokens":14,"completion_tokens":9,"total_tokens":23,"cost":0.000006}}

data: [DONE]

You pay only for the tokens produced before the failure. If the provider reported usage, that is what we charge; if it did not, we estimate it from your prompt and the output streamed so far (about four characters per token), never more than the request's hold. If it did not report usage and no output was produced, nothing is charged.

A stream holds only the prompt plus its first 8,192 output tokens at the start, and the hold grows as output arrives. If the next part can't be covered, the stream ends with an error chunk, and you pay only for the output already sent. Its error.type says why:

• insufficient_credits (code 402): your balance;
• key_limit_exceeded (402): one of the key's spending limits, named in error.limit (total, daily or monthly, with error.resets_at for the last two);
• invalid_api_key (401): the key was disabled, revoked or expired while streaming.

Usage is always included

You do not need stream_options.include_usage: every stream that is not cancelled ends with the usage chunk, whatever you set.

Cancelling

Closing the connection cancels the request and closes our connection to the provider. You are charged for the tokens generated up to that point, estimated from the output produced if the provider had not yet reported usage.