Skip to main content

Streaming

client.chat.stream() returns an Iterator[str] that yields text deltas as they arrive.

Sync streaming​

for chunk in client.chat.stream(
    model="auto",
    messages=[{"role": "user", "content": "Write a haiku about the sea."}],
):
    print(chunk, end="", flush=True)

print()  # newline after stream ends

Collecting the full text​

full_text = "".join(
    client.chat.stream(
        model="auto",
        messages=[{"role": "user", "content": "Tell me a story."}],
    )
)
print(full_text)

Async streaming​

The async client exposes the same stream() method as an AsyncIterator[str]:

from nuroute import AsyncNuRouteClient

async def main():
    async with AsyncNuRouteClient(api_key="aicp-...", base_url="https://your-gateway") as client:
        async for chunk in client.chat.stream(
            model="auto",
            messages=[{"role": "user", "content": "Hello!"}],
        ):
            print(chunk, end="", flush=True)

Parameters​

stream() accepts the same parameters as complete(). See Chat completions.

The iterator yields text only, so the SDK does not expose the gateway's X-Router-* response headers. To see whether Artemis Task or fallback-router:apollo decided a streamed request, and why, find it in the dashboard's Request Explorer or with client.requests.list() (see Which router decided).