Routing with Artemis Task

The NuRoute gateway picks a model and a provider for every request, and logs each decision so you can audit it later.

• You name a model (model: "gpt-5.5"): no router runs. Your strategy ranks that model's providers.
• You send model: "auto": Artemis Task picks the model for this prompt, then the provider. If it cannot decide, fallback-router:apollo decides instead, and the response says why (below).

const res = await client.chat.complete({
  model:    'auto',
  messages: [{ role: 'user', content: 'Summarize this in one line: ...' }],
});

How well Artemis Task chooses is measured in Benchmarks: Artemis Task.

How Artemis Task decides

For every model in its pool, Artemis Task estimates how well that model would answer this prompt and what the answer would cost. It then picks by your preset:

1. Profile the prompt. A small fine-tuned text encoder reads view the request. It is not a generative model. It returns the prompt's mix of task types: a weight for each of 24 types such as extraction, summarisation, coding, translation or reasoning.
2. Expected score per model. For every model in the pool, the capability matrix gives the expected score of its answer to a prompt with that mix.
3. Expected cost per model. A second small model predicts how long each model's answer will be. Artemis Task prices the prompt's tokens plus that answer at the catalogue price.
4. Apply the preset. Your cost and latency caps, if set, first rule out each model expected to cost more, or take longer to answer, than they allow for this prompt. The preset turns the remaining scores and costs into one choice. With Speed on, each model's expected time to answer counts too.
5. Pick the provider. Your governance rules (blocked providers, regions, provider types and so on) have already removed the providers you do not allow, and the request's needs (context size, tool calling) the ones that cannot serve it. Providers failing their health checks, and models their provider has stopped serving (checked daily), are left out too; each exclusion is recorded with its reason. Among the providers left for the chosen model, your provider preference wins (your preferred providers, then your provider order), then lower latency (observed, else the catalogue's estimate). Under Fastest, model and provider are chosen together by expected finish time.

Ties go to the cheaper model, then the one served by the provider you prefer, then the model id. Cost, latency and quality controls sets out what each setting does, the cost and latency caps included.

Artemis Task decides every auto request

Artemis Task decides every model: "auto" request, and fallback-router:apollo decides only the requests Artemis Task cannot (below). Neither your organization nor a project has a Router setting, in the dashboard or through the API; a router setting that an older client still sends is ignored.

• model takes "auto", or a model id to name one. "auto-artemis-task" and "auto-v2" mean the same as "auto" and stay accepted. "auto-apollo-tier" and "auto-v1", which used to choose the fallback router for one request, are refused with 400 invalid_request_error: the fallback router cannot be chosen.
• The x-nuroute-router header is not needed. artemis or v2 is accepted and changes nothing; apollo, v1 or any other value is refused with 400 invalid_request_error.
• GET /v1/models lists real models only, not auto or its synonyms.

With model: "auto", the response's model is the model that served, in every streamed chunk too. A request that names a model uses no router. Artemis Task chooses only among the models in its pool; to use another model from the catalogue, name it in model.

Self-hosted NuRoute does not include Artemis Task yet. There, fallback-router:apollo decides every auto request: X-Router-Version is v1 and X-Router-Fallback is v2_disabled.

Strategies and presets

Your routing strategy selects Artemis Task's preset. When you name a model, no router runs, and the strategy ranks that model's providers. Artemis Task does not read the custom cost / latency / quality weights of balanced: they rank the providers of a model you name, and the fallback router's choice. For latency, Artemis Task has the Speed setting.

Presets

manual has no preset: your pinned provider serves those requests, recorded with the reason manual_mode (fallback).

Preset	Strategy setting	How it picks	Quality target
Quality-first	highest_quality	The best expected score; extra cost counts for little	99%
Balanced	balanced (default)	The best expected score after a moderate penalty for extra cost	97%
Cost-first	cheapest	The best expected score after a heavy penalty for extra cost	92%
Fastest	fastest	On a capability version with a speed table: Balanced at speed Fastest (below). On one without: among the models whose expected score is within a set margin of the best for this prompt, the one expected to finish its answer soonest; then the cheaper	97%

The penalty for extra cost. Quality-first, Balanced and Cost-first subtract from each model's expected score a penalty that grows with how much more the model is expected to cost than the cheapest candidate for the same request. The penalty's weight is the only difference between the three presets.

Expected finish time. Fastest and the Speed setting multiply each model and provider's observed latency per output token by the model's expected answer length. They use the catalogue's latency estimate until there is enough traffic to observe.

The quality target. A capability version's preset settings are fitted on NuRoute's training data: each preset's penalty weight (Fastest's margin), and the weight again for each speed. The fit makes the preset keep at least the target share of the quality of the best single model in the pool on those prompts. They are not a guarantee on your traffic.

Setting the strategy

Set the strategy in the dashboard (Settings → Routing; for a project, Projects → your project → Routing) or through the API: PUT /v1/routing/config for your organization, PUT /v1/projects/{id}/routing/config for a project. The organization's PUT replaces the whole configuration, so start from the current one. It keeps speed and the model aliases when you leave them out, and ignores aliases sent in its body: manage them with PUT /v1/routing/aliases and DELETE /v1/routing/aliases/{alias}. A project's PUT keeps every field you leave out. If the stored configuration cannot be read at that moment, the organization's PUT, and a project's that leaves a field out, is refused with 503 and saves nothing. Changing either needs an owner's or admin's session: the dashboard, or in the SDKs a token from auth.login(). An API key can read the configuration, and a change made with it is refused with 403.

// JavaScript
const config = await client.routing.getConfig();
await client.routing.updateConfig({ ...config, strategy: 'balanced' });

# Python
config = client.routing.get_config()
client.routing.update_config({**config, "strategy": "balanced"})

Speed

Your strategy sets the quality target, through its preset. A second setting, Speed, says how much a quicker answer is worth: Off, Faster or Fastest. The preset keeps its quality target at every speed and pays for a quicker answer in dollars.

With speed on, each model's expected score is lowered for each doubling of its expected time to answer this prompt over the quickest candidate's: by 0.01 at Faster and 0.05 at Fastest, on the 0-to-1 score scale. The expected time to answer is the model's latency per output token (observed, else the catalogue's estimate) times its expected answer length. For each preset and speed, the speed table refits the penalty for extra cost so that the preset still keeps its quality target on the training prompts: a quicker answer is paid for in dollars, not in the target.

The first version with a speed table, measured in cross-validation on NuRoute's 1,049 training prompts. Each cell is dollars per 100k requests, then the models' answer time at the median and at the 90th percentile, as timed when the training answers were generated in batches. These figures are not a guarantee on your traffic.

Preset (quality target)	Off	Faster	Fastest
Balanced (97%)	$135 · 11.4 s · 97.2 s	$145 · 7.5 s · 63.0 s	$174 · 5.8 s · 25.2 s
Cost-first (92%)	$98 · 22.4 s · 93.5 s	$101 · 14.4 s · 83.7 s	$118 · 6.6 s · 49.4 s
Quality-first (99%)	$1,514 · 7.3 s · 32.8 s	$1,782 · 6.5 s · 27.8 s	$1,933 · 6.5 s · 27.6 s

On the same prompts the Fastest preset, at its 97% target, costs $334 per 100k requests, with answer times of 5.7 s and 22.7 s. Speed concentrates traffic on the quicker models.

• Dashboard (on Cloud; the self-hosted dashboard has no Speed setting): Settings → Routing → Configuration, the Speed card. It shows in auto mode, with the active version's table for your strategy's preset. For a project: Projects → your project → Routing, Speed (Artemis Task).
• API: speed on PUT /v1/routing/config ("off", "faster", "fastest", or null for off), and router_speed on PUT /v1/projects/{id}/routing/config ("" follows the organization). Both PUTs keep the setting when the body leaves it out, and any other value is a 400. GET /v1/routing/speed returns the active capability version's table.

const config = await client.routing.getConfig();
await client.routing.updateConfig({ ...config, speed: 'faster' }); // null: off

If the version in use has no speed table, or none for your preset at that speed, Artemis Task decides at speed Off and the decision's reasons include speed_unavailable:<speed>; the fallback router does not take over.

The capability matrix

The capability matrix is what Artemis Task knows about the models in its pool. For each model and each of the 24 task types it holds the expected score of the model's answers on prompts of that type, with an uncertainty band (5% to 95%). For a prompt that mixes task types, the same fit gives the expected score for that mix. Artemis Task decides on the expected score, and the band shows how sure the fit is. These scores are measured, not taken from vendors: each pool model answers NuRoute's exam prompts, the answers are graded, and the grades are fitted per task type.

Which router decided

Every model: "auto" response says which router decided it, streamed or not.

Header	Meaning
X-Router-Version	v2 when Artemis Task decided, v1 when fallback-router:apollo did
X-Router-Capability-Version	Artemis Task only: the capability version it decided with, e.g. cap-2
X-Router-Fallback	Present when the fallback router or manual routing decided instead of Artemis Task; the value is the reason
X-Router-Model	The model served
X-Router-Estimated-Saving	Estimated fraction saved against always using NuRoute's large reference model, from the served model's price
X-Router-Tier, X-Router-Confidence, X-Router-Tier-Escalated	Only when fallback-router:apollo decided: the tier it predicted (0 small · 1 medium · 2 large), its probability for that tier, and the higher tier it served, if it served one

The tier headers are not sent under manual routing, where your pinned provider serves and no tier is used. When a provider fails and the request is routed again, the headers describe the route that served the answer.

The same facts are in the response body and in the request log:

• Response body: with "debug": true in the request, the _routing object has routerVersion, capabilityVersion and routerFallback (empty strings when they do not apply), and confidence only when the fallback router chose by tier.
• Request log: the dashboard's Request Explorer shows Artemis Task · cap-2, fallback-router:apollo or Manual routing, and why the fallback router decided. It also shows each candidate's expected score, and its utility or, under Fastest, its expected latency. Through the API, GET /v1/requests/{id} and GET /v1/routing/history/{id} (client.requests.get(id), client.routing.getDecision(id)) return routerVersion, capabilityVersion and routerFallback. All three are null for a request that named a model.

Request handling

Prompt text is redacted before it reaches Artemis Task, and Artemis Task makes no network calls of its own. No LLM sits in the routing hot path. Its two small models, the task-type encoder and the answer-length model, run on every auto request.

	Behaviour
Streaming	Supported. Routing is resolved before the stream opens, so a routing failure still returns a normal JSON error rather than a half-written stream. Streamed responses carry the same X-Router-* headers as non-streamed ones.
Multi-turn	Artemis Task profiles the system prompt and the latest user message only, so a short follow-up ("now in French") is judged on its own text. The whole conversation still counts towards the token estimate and so the expected cost. The model is decided per request, so a conversation can move between models.
Tool calling	Supported, with a provider-level check: a request carrying tools goes only to providers that declare function calling. The capability matrix does not measure how reliably a model calls tools, so on quality-sensitive tool traffic an explicit model is the safer choice.
Structured output	response_format is not supported yet. The field is ignored rather than honoured, and routing does not take it into account. To get JSON reliably today, name a model you know supports it and instruct the model in the prompt.

When something fails

Situation	What NuRoute does
Artemis Task cannot decide: no estimates in time (or no active capability version), an unknown capability version or invalid estimates, manual routing or a pinned provider, no preset for your strategy in this version, no servable pool model, an error while deciding, or Artemis Task switched off	fallback-router:apollo decides the request. The response carries X-Router-Fallback with the reason.
Your policy excludes every provider	Applies your fallbackBehavior: fail (422 no_available_model), or relax to commercial / global / cheapest. The relaxation is recorded on the decision. A project's policy and provider connections are never relaxed: 422.
Your cost or latency cap rules out every model for this prompt	fail refuses the request. Any other fallbackBehavior lets the fallback router decide it under its per-provider checks; the decision's reasons include policy_caps_exhausted, and its excluded list carries what the caps ruled out, with the reason over_max_cost or over_max_latency.
A provider rate-limits, times out, goes down, rejects credentials, or no longer knows the model	Excludes that provider and decides again with the same estimates: the preset's next-best (model, provider), up to three attempts. Applies in auto mode only.
A request is malformed, or your policy rejects it	Surfaced immediately. These are true at every provider, so retrying would spend latency to fail identically and bury the real error.
Every provider is rate-limited	Returns 429 rather than hanging.

Cross-provider retry applies to non-streaming requests in auto mode. A streaming response that fails after the first token has already been sent is surfaced as a stream error rather than silently re-routed. And retry is a failover mechanism, not a load balancer: NuRoute reacts to a failure rather than tracking per-provider rate-limit budgets in advance.

Fallback (fallbackBehavior)

fallbackBehavior controls what happens when no provider passes your governance rules (blocked providers, region restrictions, cost or latency limits and so on). It runs before the router chooses, and works the same whichever router decides. It is neither a retry-on-5xx setting nor the fallback router.

Value	Behaviour
fail	Refuse the request if no eligible provider is found: 422, type no_available_model
allow_commercial	Fall back to any enabled commercial provider (default)
allow_global	Fall back to any enabled provider globally
allow_cheapest	Fall back to the cheapest available provider

Routing history

Every routed request creates a routing decision record. You can list decisions and inspect why a particular model and provider were chosen:

const { decisions } = await client.routing.listHistory({ limit: 10 });

const detail = await client.routing.getDecision(decisions[0].id);
// detail.selectedProvider, detail.selectedModel, detail.candidates, detail.strategy,
// detail.routerVersion, detail.capabilityVersion, detail.routerFallback

On an Artemis Task decision, each candidate also carries quality (the model's expected score), utility (null under Fastest), inputTokens, outputTokens and expectedLatencyMs.

See also

• Cost, latency and quality controls: what strategy, weights, speed and the cost and latency caps do, and what to change to make routing cheaper or faster
• Benchmarks: Artemis Task: RouterArena's independent benchmark, and our enterprise test against single models and by industry
• Projects: settings per project, and project-scoped API keys