Skip to main content

LangChain connector

langchain-nuroute is the official LangChain chat model for NuRoute. It plugs a NuRoute-routed model into any LangChain chain, agent, or RunnableSequence as ChatNuRoute — a normal BaseChatModel, so it composes with everything else LangChain has (prompts, output parsers, tools, LCEL pipes) exactly like ChatOpenAI or ChatAnthropic would.

How it works​

ChatNuRoute calls your NuRoute gateway through the nuroute Python SDK client, not through an OpenAI-compatibility shim. Pointing LangChain's own ChatOpenAI at the gateway's OpenAI-compatible /v1/chat/completions endpoint also works — the gateway accepts OpenAI-shaped requests directly — but it flattens NuRoute-specific behavior into OpenAI's response and error shapes. Going through ChatNuRoute instead keeps that behavior intact: the "auto" routing model, provider overrides to pin a specific upstream provider, and gateway errors surfacing as NuRouteError rather than an OpenAI-shaped exception.

Install​

pip install langchain-nuroute

Requires Python 3.9+.

Quick start​

from langchain_nuroute import ChatNuRoute
from langchain_core.messages import HumanMessage

llm = ChatNuRoute(
    nuroute_api_key="aicp-...",
    nuroute_base_url="https://your-nuroute-gateway",
    model="auto",  # NuRoute's routing engine picks the best model — see Concepts > Routing
)

response = llm.invoke([HumanMessage(content="Hello!")])
print(response.content)

With model="auto", Artemis Task picks the model for each request.

ChatNuRoute is a normal BaseChatModel, so it works in a chain the same way any other LangChain chat model would:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_template("Summarize this in one sentence:\n\n{text}")
chain = prompt | llm

print(chain.invoke({"text": "..."}).content)

Streaming​

for chunk in llm.stream([HumanMessage(content="Tell me a short story")]):
    print(chunk.content, end="", flush=True)

Forcing a provider or model​

Combine a specific model with provider to pin both. With model="auto" and a provider, no router runs: the provider serves the first model it lists.

llm = ChatNuRoute(
    nuroute_api_key="aicp-...",
    nuroute_base_url="https://your-nuroute-gateway",
    model="claude-sonnet-4-6",
    provider="anthropic",
    temperature=0.2,
    max_tokens=1024,
)

Reading routing metadata​

Every response carries which model NuRoute actually routed to and the token usage for the call, available on llm_output when calling generate directly:

result = llm.generate([[HumanMessage(content="Hello!")]])
print(result.llm_output["model"])  # e.g. "gpt-4o-mini"
print(result.llm_output["usage"])  # {"prompt_tokens": ..., "completion_tokens": ..., "total_tokens": ...}

Error handling​

Gateway errors raise nuroute.NuRouteError (the same exception the SDK itself raises), with status, error_type, and code fields — not an OpenAI-shaped exception:

from nuroute import NuRouteError

try:
    llm.invoke([HumanMessage(content="Hello!")])
except NuRouteError as e:
    print(e.status, e.error_type, e.message)

What's not supported​

ChatNuRoute doesn't accept a stop parameter — the gateway's /v1/chat/completions endpoint has no stop-sequence support today, so passing one raises NotImplementedError rather than silently ignoring it.