Question 1

What is NuRoute?

Accepted Answer

NuRoute sits between your application and AI inference providers. You make one API call, and it handles model routing, failover, cost tracking, and policy enforcement instead of you wiring up OpenAI, Anthropic, Gemini, and every other provider separately.

Question 2

How is this different from calling a provider's API directly?

Accepted Answer

Calling a provider directly means you own the failover logic, the cost tracking, and the policy enforcement, and you own it again for every provider you add. NuRoute gives you one OpenAI-compatible endpoint: it picks the model, retries across providers on failure, logs every decision, and enforces your org's policies. Add a provider and nothing in your application changes.

Question 3

Is NuRoute open source?

Accepted Answer

No. NuRoute is closed-source. Cloud is self-serve with prepaid credits, and Enterprise supports a self-hosted deployment under a commercial license, but the source itself isn't publicly available.

Question 4

How many providers and models does NuRoute support?

Accepted Answer

23 provider integrations as of writing: 10 commercial (OpenAI, Anthropic, Google Gemini, Mistral, Cohere, xAI, AWS Bedrock, AWS SageMaker, Azure OpenAI, Azure AI Foundry), 5 European/GDPR-native (Nebius, OVH, Scaleway, Aleph Alpha, IONOS), 2 fast-inference platforms (Groq, Cerebras), and 6 ways to connect a self-hosted model (vLLM, Ollama, Hugging Face TGI, NVIDIA NIM, SGLang, or a generic custom endpoint). The catalog of models behind those providers changes as providers add, reprice, or deprecate models, so any specific model count goes stale fast. The in-product Model Catalog is the live source of truth.

Question 5

How many of those models can automatic routing actually pick?

Accepted Answer

Artemis Task, which decides your model: "auto" requests, picks only among the models in its active capability version's pool: A multitude of models today, each measured on graded answers to our exam prompts. You can still select any catalogued model explicitly by name; the restriction is only on automatic routing. If Artemis Task can't decide a request, fallback-router:apollo decides it and may pick a catalogued model outside the pool; the response says why. See Capability versions for the pool and how a model joins it.

Question 6

How quickly does a newly released model become available?

Accepted Answer

There's no service commitment on timing. A new model is usually selectable by name as soon as its provider lists it. Getting it into automatic routing takes longer, because it has to be measured first: it sits an exam (it answers a few hundred exam prompts and an LLM judge grades them), and Artemis Task can pick it once a new capability version that includes it has passed its checks and been activated. See Capability versions.

Question 7

What is BYOK?

Accepted Answer

Bring Your Own Key. Connect your own provider credentials and NuRoute uses them for real inference right away. Your own key always takes priority over any provider your admin has connected platform-wide, for that same provider. BYOK works on every plan, including self-hosted Enterprise, and keys are AES-256-GCM encrypted before storage.

Question 8

Do I have to list which models a provider supports?

Accepted Answer

No. Once you connect a key, NuRoute discovers that provider's available models automatically from its own model-listing endpoint.

Question 9

Why do some providers need the dashboard instead of the SDK?

Accepted Answer

Most providers authenticate with a single API key, which client.providers.add() handles directly. Four don't: AWS Bedrock, AWS SageMaker, Azure OpenAI, and Azure AI Foundry authenticate with structured, multi-field credentials (SigV4 keys, resource or deployment names, API versions) instead of a single key, and self-hosted or custom endpoints need a base URL. Both are configured from the dashboard's Providers tab today; the SDKs don't expose multi-field credentials yet.

Question 10

Can I point NuRoute at my own model server?

Accepted Answer

Yes. vLLM, Ollama, Hugging Face TGI, NVIDIA NIM, and SGLang are named runtimes, and any endpoint that speaks the OpenAI chat completions API can be connected as a custom endpoint. Whatever your deployment serves becomes available to route to.

Question 11

Which router decides my model: "auto" requests?

Accepted Answer

Artemis Task decides every one: it estimates each pool model's score and cost for the prompt and picks by your strategy's preset. There is no router to choose and no Router setting, for your organization or for a project. A request that names a model uses no router. If Artemis Task can't decide a request, fallback-router:apollo decides it and the response says why, in X-Router-Fallback. The fallback router can't be chosen: a request that asks for it is refused with a 400. See Routing.

Question 12

What actually makes the routing decision: fixed rules, a call to an LLM, or a trained model?

Accepted Answer

Trained models. Artemis Task uses no hand-written rules and calls no large language model to make the decision. A small model of its own reads the start of the system prompt and the start and end of the latest message, and estimates the prompt's mix of task types. The capability matrix, each pool model's measured expected score per task type, turns that mix into an expected score for every model; a length model predicts each answer's length, which with catalogue prices gives an expected cost. Your preset's rule then picks. If Artemis Task can't decide in time, fallback-router:apollo does. We explain how the matrix is built on the Routing page; we don't publish its per-model scores.

Question 13

What does model: "auto" do?

Accepted Answer

It hands model selection to NuRoute instead of you naming a model, and Artemis Task makes the choice. For every model in its pool, it estimates the score its answer would get and what it would cost, then picks by your strategy's preset: balanced → Balanced, highest_quality → Quality-first, cheapest → Cost-first, fastest → Balanced at speed Fastest (the Fastest preset on a capability version without a speed table). manual has no preset: your pinned provider serves the request, recorded as manual_mode. Quality-first, Balanced and Cost-first pick the best expected score after a price for extra cost; the more a preset leans toward cost, the more quality an extra dollar has to buy. The Fastest preset keeps the models whose expected score is within a small margin of the best and picks the one expected to answer soonest. A separate Speed setting (Off, Faster, Fastest) makes a quicker answer count while the preset keeps its quality target. Your governance rules, enabled providers, provider preference (your preferred providers, then your provider order) and latency then decide the provider. If Artemis Task can't decide a request, fallback-router:apollo does, and the response says why.

Question 14

How much does automatic routing actually save?

Accepted Answer

On RouterArena, an independent public benchmark of 8,400 questions graded by fixed rules, Artemis Task answered 75.5% correctly at $0.60 per 1,000 queries. GPT-5 answering every question scored 74.0% at $10.02, so Artemis Task was more accurate at 6% of the cost. On our own pre-registered test of 426 held-out business prompts, graded by an LLM judge (gemini-3.8-flash), Artemis Task at Balanced matched Claude Sonnet 5's quality (0.952 against 0.949) at 3.5% of its cost, and kept 97.5% of gpt-5.5's quality at 2.5% of its cost. That test favours Artemis Task: we built it from the same kind of prompts it was trained on, and on these business tasks cheap models score close to expensive ones, so read it as the best case. Most of the saving comes from not sending everything to the most expensive model. Neither test is your traffic. See Benchmarks: Artemis Task.

Question 15

How does NuRoute measure whether a model is "good enough"?

Accepted Answer

We measure it ourselves rather than take a vendor's marketing claims. Artemis Task measures each pool model on graded answers to exam prompts: the model answers them and an LLM judge grades the answers (a new model sits an exam of a few hundred prompts). The grades are fitted into the capability matrix: per model and per task type (24 of them), the expected score of its answers with an uncertainty band. The matrix ships as a capability version, published once and never changed; one version is active at a time, and rolling back means activating the previous one. We explain how it is built on the Routing page and don't publish the per-model scores.

Question 16

Can we see the routing decision for each request?

Accepted Answer

Yes. Responses, streamed or not, carry headers: X-Router-Version (the router that decided: v2 for Artemis Task, v1 for fallback-router:apollo), X-Router-Capability-Version (the capability version, when Artemis Task decided), X-Router-Fallback (why the fallback router decided instead of Artemis Task), X-Router-Model (the model that served it) and X-Router-Estimated-Saving. X-Router-Tier, X-Router-Confidence and X-Router-Tier-Escalated are sent only when the fallback router chose by tier; Artemis Task's decisions and manual routing don't carry them. If a request is routed again after a provider failure, the headers describe the route that served the answer. Every decision is also recorded: the Request Explorer, GET /v1/requests/:id and GET /v1/routing/history/:id show routerVersion, capabilityVersion and routerFallback, and a chat request sent with debug: true returns the same fields in _routing. See Which router decided.

Question 17

Does the router send our prompt content to a third party to make its decision?

Accepted Answer

No. Routing runs inside NuRoute's own infrastructure (yours, if you're self-hosting). Artemis Task and its fallback router, fallback-router:apollo, hold no provider API keys, make no outbound calls of their own, and never call a language model API to decide; Artemis Task's small text models run there too. Your prompt only ever travels to whichever provider your routing decision selects, the same provider you'd be calling if you integrated with them directly.

Question 18

How much latency does routing itself add?

Accepted Answer

Artemis Task runs small text models on the prompt: in our tests on a laptop CPU, its decision took 127ms at the median and 161ms at p95. We haven't published a figure for production hardware yet. That time comes on top of whatever the chosen provider takes to answer, not instead of it. If Artemis Task's estimates aren't ready in time, fallback-router:apollo decides the request and X-Router-Fallback says so. How long the chosen model takes to answer is a separate matter: if that matters most, the fastest strategy and Artemis Task's Speed setting are what optimise for it (see I want faster answers).

Question 19

Do you rank models on published benchmarks or on speed you measure yourselves?

Accepted Answer

For speed, every model starts with a benchmark-derived latency estimate. Once a model has accumulated enough recent requests on your own traffic, the observed figure replaces that estimate, so ranking reflects how each model actually performs for you rather than a generic benchmark. Artemis Task uses it for its Fastest preset, its Speed setting and to choose the provider, and the fastest and balanced strategies use it to rank the providers of a model you name. Quality comes from our own measurements, not published benchmarks (see the question on how a model is judged "good enough").

Question 20

What routing strategies can I choose from?

Accepted Answer

Five: cheapest, fastest, highest_quality, manual (pins a specific provider you choose), and balanced, the default. When you name a model directly, the strategy decides which of its providers wins. For auto requests, Artemis Task uses it to select a preset (balanced → Balanced, highest_quality → Quality-first, cheapest → Cost-first, fastest → Balanced at speed Fastest, or the Fastest preset on a capability version without a speed table; under manual, your pinned provider serves the request, recorded as manual_mode). Custom balanced weights rank the providers of a model you name; Artemis Task doesn't read them and has a Speed setting instead. See Routing for configuration, and Cost, latency and quality controls for what each setting does.

Question 21

How do I make routing cheaper or faster?

Accepted Answer

With your strategy and the Speed setting. The strategy sets Artemis Task's quality target (cheapest → Cost-first keeps 92% of the best model's quality, balanced 97%, highest_quality 99%), the Speed setting buys quicker answers in dollars at the same target, and the cost and latency caps are checked per request. Cost, latency and quality controls has worked examples.

Question 22

What happens if no eligible provider can serve a request?

Accepted Answer

It depends on fallbackBehavior. fail refuses the request (422, type no_available_model), allow_commercial (the default) falls back to any enabled commercial provider, allow_global falls back to any enabled provider, and allow_cheapest falls back to whatever's cheapest available. This only fires when your policy excludes every eligible provider, or when your cost and latency caps rule out every model Artemis Task could pick for the prompt; then any value but fail lets fallback-router:apollo decide the request. A project's policy and provider connections are never relaxed: when nothing they allow can serve the request, it gets 422 no_available_model whatever the value. Provider failures are handled separately, by re-routing and retrying across providers automatically.

Question 23

Can I override the router: pin a model, block providers, restrict regions?

Accepted Answer

Yes. Name a specific model to bypass automatic selection, pin or block particular providers, restrict which regions may serve your requests, cap the cost per request and the latency, and exclude whole provider categories such as open-source or self-hosted. These are enforced as a hard governance filter that runs before any routing choice is made, whichever router decides, and when you name a model yourself. The cost and latency caps are the exception in how they are checked: Artemis Task checks them per request on each model's expected cost and time to answer; for a model you name, and when fallback-router:apollo decides, they are checked per provider (see Cost, latency and quality controls). Unless your fallbackBehavior is fail, your organization's rules that exclude every provider are relaxed rather than failing the request; a project's policy never is (see Fallback).

Question 24

What are model aliases?

Accepted Answer

Names like "smart" or "fast" that you map to real model ids once, at the org or project level. Chat requests don't resolve them yet, so for now send a model id or "auto" as model (see Create or update an alias).

Question 25

What languages does the router support?

Accepted Answer

Artemis Task was tested on English and German prompts only; other languages are untested. On self-hosted NuRoute, fallback-router:apollo decides every auto request, and its routing model is built primarily for English. Broader language support is planned.

Question 26

What are the plans?

Accepted Answer

Cloud is self-serve with prepaid credits. It is for business use. Enterprise is self-hosted, with a custom SLA, arranged with our sales team. Pricing is being updated: book a call for current terms.

Question 27

How exactly is Cloud billed?

Accepted Answer

With prepaid credits in US dollars. You top up by card from the Billing page, and the fees that apply are shown there before you top up. With your own provider key, the provider bills you directly. A request starts only while your balance is above zero; otherwise the gateway returns 402 insufficient_credits until you top up.

Question 28

What happens if a project's budget runs out?

Accepted Answer

If the project's budget has a hard limit, the gateway refuses the project's new requests with 402 budget_exceeded once its spend this calendar month (UTC) reaches the budget, until you raise it or the month ends. The check reads spend at most 30 seconds old, so a busy project can pass its budget slightly. Without a hard limit nothing is refused. The alert threshold is saved, but alerts are not sent yet.

Question 29

Do credits expire? Can I get a refund?

Accepted Answer

Credits never expire. Unused purchased credits are refunded when you close your account. Write to us from the email address on your account.

Question 30

How are my provider API keys stored?

Accepted Answer

AES-256-GCM encrypted, per organization, in storage kept separate from the rest of the system. Keys never leave the control plane in plaintext and are never returned in readable form through any API, including to the organization that owns them.

Question 31

Does NuRoute redact personally identifiable information?

Accepted Answer

Yes, on every plan. Outbound prompts are scanned using Microsoft Presidio, and you choose whether a detection failure fails open (the request proceeds unredacted) or fails closed (the request is rejected).

Question 32

Can I control which regions my requests are processed in?

Accepted Answer

Yes. Allowed regions restrict an organization to providers in specific geographic regions, and a project can narrow its organization's regions further. A provider that declares no region (self-hosted and custom endpoints) is left out unless you allow Global; see how regions match. What happens when no provider in those regions can serve a request depends on fallbackBehavior: with fail the request is rejected; with any other value (the default is allow_commercial) your rules are relaxed and the request can be served outside them, and the decision records the relaxation. Set fail if data must never leave your regions. A project's policy can also set allowed regions, and no fallbackBehavior relaxes those. See Fallback.

Question 33

Does NuRoute train on my data?

Accepted Answer

No.

Question 34

Is NuRoute GDPR or EU AI Act compliant?

Accepted Answer

The architecture is built with both in mind: PII redaction, data residency controls, no training on your prompts or responses, and support for per-jurisdiction model filtering. ISO 27001 certification is in progress and hasn't completed yet, so we're not claiming it as done.

Question 35

What access control does NuRoute have?

Accepted Answer

Role-based, at the organization level: Owner, Admin, Member, and Viewer.

Question 36

Is there an audit log?

Accepted Answer

A partial one today. Every routing decision is already recorded and replayable through the dashboard's Request Explorer. Full audit logging with long-term retention is on the roadmap and hasn't shipped yet; see Governance & compliance for the current shipped-versus-planned breakdown.

Question 37

Can I run NuRoute entirely on my own infrastructure?

Accepted Answer

Yes, as part of the Enterprise plan. It's not a free, self-serve download — talk to us and we'll get you a licensed deployment for your own infrastructure.

Question 38

What's the difference between self-hosted Enterprise's routing and Cloud's?

Accepted Answer

On Cloud, Artemis Task decides model: "auto" requests. Artemis Task is not part of self-hosted NuRoute yet: there fallback-router:apollo decides every auto request (X-Router-Version: v1, X-Router-Fallback: v2_disabled), and the dashboard has no Speed setting. A self-hosted deployment receives that router's routing model with its licence check-in, as a stable, periodically refreshed release.

Question 39

Does the self-hosted deployment include the model-training pipeline?

Accepted Answer

No. You get the trained routing model that fallback-router:apollo decides with, plus the code that serves it, not the code or data used to train it.

Question 40

What are Projects?

Accepted Answer

Isolated workspaces within your org, available on every plan, each with its own provider connections, routing configuration, policies, budgets, environments, and API keys.

Question 41

Can different teams have different provider access?

Accepted Answer

Yes. Give each team its own project and scope its API key to that project, so it can only reach that project's providers and configuration.

Question 42

Do project settings override org-level settings?

Accepted Answer

Yes, for the project's requests: those made with a project-scoped API key, or that name the project with the X-NuRoute-Project-ID header. Other requests use the organization's settings. Governance stays the organization's: blocked and preferred providers, cost and latency caps, provider types and fallbackBehavior apply to every project, and a project's allowed regions can only narrow the organization's. A project's provider order, affinity and balanced weights follow the organization's until the project sets its own.

Question 43

What SDKs are available?

Accepted Answer

Official JavaScript/TypeScript and Python SDKs, both with synchronous clients. The Python SDK also has an async client.

Question 44

Is the API OpenAI-compatible?

Accepted Answer

The gateway accepts OpenAI-compatible chat completion requests, so most existing OpenAI client code can point at your NuRoute gateway URL with minimal changes.

Question 45

Is there a no-code option?

Accepted Answer

Yes, the official n8n-nodes-nuroute community node sends chat completions through NuRoute without writing any integration code. See the n8n connector guide.

Question 46

Can I use NuRoute from Claude Desktop or another MCP client?

Accepted Answer

Yes, the official nuroute-mcp-server MCP server sends chat completions through NuRoute and lists the model catalog, directly from any MCP client. See the MCP connector guide.

Question 47

Is there a LangChain integration?

Accepted Answer

Yes, the official langchain-nuroute package ships ChatNuRoute, a chat model that plugs into any LangChain chain or agent. See the LangChain connector guide.

Question 48

Is there a Vercel AI SDK integration?

Accepted Answer

Yes, the official nuroute-ai-sdk package lets generateText/streamText route through NuRoute the same way you'd use @ai-sdk/openai. See the Vercel AI SDK connector guide.

Question 49

Is there a Dify plugin?

Accepted Answer

Yes, the official NuRoute model provider plugin lets Dify apps and agents route through NuRoute directly from the model dropdown. See the Dify plugin guide.

Question 50

Does NuRoute have a CrewAI integration?

Accepted Answer

No separate package needed — CrewAI's built-in LLM class can point at any OpenAI-compatible endpoint, NuRoute included. See the CrewAI guide.

Question 51

Can I use NuRoute in a Zapier workflow?

Accepted Answer

Yes, the official NuRoute Zapier integration sends chat completions through your gateway from any Zap. It's currently private/invite-only during early access. See the Zapier connector guide.

Question 52

Does NuRoute support embeddings, images, audio, reranking, or batch processing?

Accepted Answer

Not yet. Chat completions are the only supported request type today.

Question 53

Does NuRoute support OpenAI's Responses API or Anthropic's Messages API natively?

Accepted Answer

No, to both. NuRoute accepts the OpenAI chat-completions format and translates it to each provider's native API internally. There's no Responses API and no native Messages endpoint.

Question 54

How much work is it to switch existing code over to NuRoute?

Accepted Answer

Usually just a base URL and API key change. Point your existing OpenAI-compatible client at your NuRoute gateway URL, swap in an NuRoute key, and set model to "auto" if you want automatic routing instead of naming a model directly.

Question 55

What does a 402 response mean?

Accepted Answer

Either your organization's prepaid credits are used up (type: "insufficient_credits") or the project's hard-limited monthly budget is spent (type: "budget_exceeded"). Check the error type to tell which.

Question 56

What other error types should my integration handle?

Accepted Answer

Authentication and authorization failures (401 invalid_request_error for a missing or invalid key or token; 403 forbidden, for example a change made with an API key instead of an owner's or admin's session, or email_not_verified), a model or provider the project's policy rules out (403: policy_violation), malformed requests (400/422: invalid_request_error, including a max_tokens over the project's limit), routing failures when nothing eligible could serve the request (422 no_available_model) or routing is unavailable (503 routing_error), upstream provider failures (provider_error), PII redaction being unavailable while configured fail-closed (pii_redaction_unavailable), and 429 (too_many_concurrent_requests, or provider_error when every provider is rate-limited). The JavaScript and Python error-handling guides cover the exact shape of each.

Question 57

What's coming next?

Accepted Answer

Formal governance and compliance tooling (full audit logging to ClickHouse with long-term retention, compliance reporting, certifications beyond the in-progress ISO 27001), a published REST API reference, and Kubernetes/Helm support for self-hosting. None of these have committed dates. See the individual Coming soon pages for what's shipped today versus what's still ahead.