Async client

AsyncNuRouteClient is the async counterpart to NuRouteClient. It uses httpx's async transport and exposes the same namespaces (auth, chat, providers, routing, projects, etc.) with async def methods.

Setup

from nuroute import AsyncNuRouteClient

client = AsyncNuRouteClient(
    api_key="aicp-...",
    base_url="https://your-gateway",
    timeout=30.0,
)

Context manager (recommended)

async with AsyncNuRouteClient(
    api_key="aicp-...",
    base_url="https://your-gateway",
) as client:
    response = await client.chat.complete(
        model="auto",
        messages=[{"role": "user", "content": "Hello!"}],
    )

Routing

chat.complete() and chat.stream() take the same keywords as the sync client's. Artemis Task decides every model="auto" request: router= is optional, and its only values, "artemis" and "v2", change nothing. Whether fallback-router:apollo decided a request instead, and why, is on the request log: see Which router decided.

Example: concurrent requests

import asyncio
from nuroute import AsyncNuRouteClient

async def main():
    async with AsyncNuRouteClient(api_key="aicp-...", base_url="https://your-gateway") as client:
        prompts = [
            "Summarise the French Revolution in one sentence.",
            "What is the boiling point of water in Kelvin?",
            "Name three uses of transformer models.",
        ]

        tasks = [
            client.chat.complete(
                model="auto",
                messages=[{"role": "user", "content": p}],
            )
            for p in prompts
        ]

        responses = await asyncio.gather(*tasks)

        for prompt, response in zip(prompts, responses):
            print(f"Q: {prompt}")
            print(f"A: {response['choices'][0]['message']['content']}\n")

asyncio.run(main())

FastAPI integration

from contextlib import asynccontextmanager
from fastapi import FastAPI
from nuroute import AsyncNuRouteClient

client: AsyncNuRouteClient

@asynccontextmanager
async def lifespan(app: FastAPI):
    global client
    client = AsyncNuRouteClient(api_key="aicp-...", base_url="https://your-gateway")
    yield
    await client.aclose()

app = FastAPI(lifespan=lifespan)

@app.post("/ask")
async def ask(question: str):
    response = await client.chat.complete(
        model="auto",
        messages=[{"role": "user", "content": question}],
    )
    return {"answer": response["choices"][0]["message"]["content"]}

Differences from the sync client

	NuRouteClient	AsyncNuRouteClient
Methods	def	async def
Transport	httpx.Client	httpx.AsyncClient
chat.stream()	Iterator[str]	AsyncIterator[str]
Context manager	with	async with
close()	sync	await client.aclose()