# Streaming

Server-sent events of a streamed completion, the usage chunk, errors in a stream, timeouts and what a cancelled stream costs.

With `"stream": true` the response is a `text/event-stream`: the answer arrives in chunks as the model generates it.

```bash
curl -N https://router.nodul.ru/api/v1/chat/completions \
  -H "Authorization: Bearer $NODUL_ROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "google/gemma-3-4b-it", "stream": true, "messages": [{"role": "user", "content": "Count to five."}]}'
```

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://router.nodul.ru/api/v1", api_key=os.environ["NODUL_ROUTER_API_KEY"])
stream = client.chat.completions.create(
    model="google/gemma-3-4b-it",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\ncost:", chunk.usage.cost)
```

```js
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://router.nodul.ru/api/v1", apiKey: process.env.NODUL_ROUTER_API_KEY });
const stream = await client.chat.completions.create({
  model: "google/gemma-3-4b-it",
  messages: [{ role: "user", content: "Count to five." }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  if (chunk.usage) console.log("\ncost:", chunk.usage.cost);
}
```

## Events

```text
: keepalive

data: {"id":"gen-…","object":"chat.completion.chunk","model":"google/gemma-3-4b-it","choices":[{"index":0,"delta":{"role":"assistant","content":"One"}}]}

data: {"id":"gen-…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":", two"},"finish_reason":null}]}

data: {"id":"gen-…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"gen-…","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":9,"total_tokens":21,"cost":0.0000081}}

data: [DONE]
```

- Lines that start with `:` are comments that keep the connection alive. SSE clients ignore them; a hand-written parser must skip them.
- Every chunk has the generation `id`; the response header `X-Generation-Id` carries it too.
- `delta.content` holds text, `delta.tool_calls` holds tool calls in pieces, and reasoning models may send `delta.reasoning` or `delta.reasoning_content`.
- The last chunk before `data: [DONE]` has `usage` with `cost` in PnP. You do not need `stream_options`: the router always asks for it.

## Errors in a stream

Errors before the first chunk come as a normal HTTP error with a JSON body, see [Errors](/docs/en/errors). Once the stream has started the status is already `200`, so a failure comes as an event:

```text
data: {"error":{"code":504,"message":"the upstream stream timed out (idle)"},"choices":[{"index":0,"delta":{},"finish_reason":"error"}]}
```

The router never switches to a fallback model after the first chunk: the partial answer is not regenerated elsewhere. If the connection ends without `[DONE]`, treat the answer as incomplete.

## Timeouts

| Limit                                    | Value       |
| ---------------------------------------- | ----------- |
| Longest stream                           | 15 minutes  |
| Longest pause between events             | 120 seconds |
| Wait for the provider to start answering | 120 seconds |

Use streaming for long answers and reasoning models: a non-streamed request also has to start within 120 seconds and finish within 10 minutes.

## Cancelling

Close the connection to stop generation. The request is billed for what was generated up to that moment, by the provider's usage report or, when there is none, by an estimate.