Cost, latency and quality controls

NuRoute has one set of routing settings. Artemis Task, which decides every model: "auto" request, turns them into a quality target and a price on cost and time, and weighs every model in its pool against them. Some settings also apply to requests that name a model, and to fallback-router:apollo, which decides only the requests Artemis Task cannot (when); each section says where.

At a glance

Control	What it does
Strategy	Selects a preset, and the preset sets a quality target: Quality-first 99%, Balanced 97%, Cost-first 92%. On a model you name, it ranks that model's providers
Balanced weights	Not read by Artemis Task. Under balanced, they rank the providers of a model you name, and the fallback router's choice
Speed	Off, Faster or Fastest: quicker answers at the same quality target, paid for in dollars
Max cost per request	Checked per request, on each model's expected cost for this prompt. Named models and the fallback router: a rough check per provider
Max latency	Checked per request, on each model's expected time to answer this prompt. Named models and the fallback router: a check per provider
Provider order	Picks among the providers of the chosen model, and breaks ties. Named models and the fallback router: a small boost and the tie-break
Governance and fallbackBehavior	Applied before the router chooses, and to named models too
Manual mode, or a pinned provider	No preset: your pinned provider serves the request (manual_mode)

Each of these is a setting of your organization, and some of a project (where to set each). No request header or body field sets them (per request).

Governance and fallback, before the router chooses

These rules run on the list of providers before the router chooses, whichever router decides:

• Blocked providers: never used.
• Allowed regions: only providers that serve one of these regions are used. Empty allows all. See How regions match.
• Provider types: open-source, proprietary, self-hosted and managed inference platforms, each allowed or not.

They apply to requests that name a model too. fallbackBehavior says what happens when they rule out every provider:

Value	When your rules rule out every provider
fail	The request is refused: 422, type no_available_model
allow_commercial (default)	Any enabled commercial provider may serve it
allow_global	Any enabled provider may serve it
allow_cheapest	The cheapest enabled provider serves it

Every value except fail relaxes your rules in that case, regions included. Set fail if a request must never reach a provider your rules exclude. More in Fallback.

The cost and latency caps belong to governance too, but Artemis Task checks them per request; they have their own sections below.

How regions match

Region names are not case-sensitive: EU and eu are the same. Each region group covers its own code and the cloud regions inside it:

You allow	A provider passes when it declares
EU	eu, de, fr, or a region starting eu- (eu-central-1)
US	us, or a region starting us-
DE	de, or a region starting eu-central-
FR	fr, or a region starting eu-west-
APAC	apac, or a region starting ap- or as-
Global	any region, so it lifts the restriction
any other name	exactly that region

The dashboard's chips save eu, us and global. Each provider's regions are the ones on its card under Providers. A provider that declares no region (self-hosted and custom endpoints such as vLLM or Ollama) has an unknown location, so under any list without Global it is excluded. To use a self-hosted provider that runs in the EU under an EU restriction, its region has to be declared for it: ask your operator (on a self-hosted installation, the operator sets it in routing-service's provider registry), or allow Global.

When no router chooses

• You name a model (model: "gpt-5.5"). No router runs, and your strategy ranks that model's providers: balanced by cost and latency under your weights (quality is the same for every provider of one model), cheapest by the estimated cost of a request, fastest by typical latency, highest_quality by NuRoute's provider priority. The caps are checked per provider (max cost, max latency). Speed does not apply.
• Manual mode. With mode manual (or the manual strategy), your pinned provider and model serve every auto request. Artemis Task has no preset for manual, so manual routing serves the request and X-Router-Fallback says manual_mode.
• A request that pins a provider is routed manually for that request. With model: "auto", no router picks the model: the provider serves its default model (the first it lists). Name a model as well to choose it.

Strategy

Set it: dashboard Settings → Routing → Configuration, Default Strategy; for a project, Projects → your project → Routing, Strategy. API: strategy on PUT /v1/routing/config, or on PUT /v1/projects/{id}/routing/config. Values: balanced (default), cheapest, fastest, highest_quality, manual.

The strategy selects Artemis Task's preset, and a preset is a quality target: the share of the best single model's quality that the preset must keep on NuRoute's training data.

Strategy	Preset	Quality target
highest_quality	Quality-first	99%
balanced	Balanced	97%
cheapest	Cost-first	92%
fastest	Balanced at speed Fastest, on a capability version with a speed table (cap-4 and later). On an older version, the Fastest preset	97%
manual	None: manual routing serves the request

Ties go to the cheaper model, then your provider order, then the model id. The Fastest preset, used by fastest on a version without a speed table, works differently: it keeps the models whose expected score is within a fitted margin of the best, and picks the one expected to finish its answer soonest. See Artemis Task: Presets.

So changing the strategy moves you between quality targets, prompt by prompt across the whole pool. On a request that names a model, the strategy ranks that model's providers instead (above), and the fallback router ranks by it too when it decides.

Speed

Set it: dashboard Settings → Routing → Configuration, the Speed card (shown in auto mode, on Cloud); for a project, Projects → your project → Routing, Speed (Artemis Task). API: speed on PUT /v1/routing/config ("off", "faster", "fastest", or null for off) and router_speed on PUT /v1/projects/{id}/routing/config ("" follows the organization). A project's speed applies to its requests, else the organization's, else Off.

Speed adds a term for time to the preset's score: a quicker answer at the same quality target, paid for in dollars (Artemis Task: Speed). Requests that name a model, and the fallback router, ignore it.

Max cost per request

Set it: dashboard Settings → Routing → Organization Policies, Cost & Latency Limits, Max cost per request (USD). API: policy.maxCostPerRequest on PUT /v1/routing/config, in US dollars; leave it out for no limit. It is the organization's, for every project.

Artemis Task checks it per request, for each model: the model's expected cost for this prompt (input tokens at its input price, plus its expected answer length at its output price). A model expected to cost more than the cap is not chosen; the others stay in. The decision's excluded list names each model and provider the cap ruled out (over_max_cost), with the expected cost against the cap.

On a request that names a model, and when the fallback router decides, it is a rough check per provider instead. A provider is left out when its listed price per 1M input tokens is above the cap × 1,000; the check assumes a request of about 1,000 input tokens. It does not look at your prompt, the answer's length or output prices, and it uses one listed price per provider, whatever the model. A cap of $0.01 leaves out every provider listed above $10 per 1M input tokens.

Max latency

Set it: dashboard Settings → Routing → Organization Policies, Cost & Latency Limits, Max latency (ms). API: policy.maxLatencyMs on PUT /v1/routing/config; leave it out for no limit. It is the organization's, for every project.

Artemis Task checks it per request, for each model on each provider: the expected time to finish the whole answer to this prompt (as in Speed). A pair expected to take longer is not chosen (over_max_latency in the excluded list).

On a request that names a model, and when the fallback router decides, a provider is left out when its listed typical latency is above the cap. That is one figure per provider, not per model, and not observed from your traffic.

Set it for the whole answer

Artemis Task compares the cap with the expected time to finish the whole answer, which is often several seconds (see the answer times in Artemis Task: Speed). A cap sized for a provider's typical latency figure, which is usually under a second, rules out most models.

When the caps rule out every model under Artemis Task

If your cost and latency caps rule out every model Artemis Task could serve for a prompt:

• with fallbackBehavior: "fail", the request is refused: 422, type no_available_model;
• with any other value, fallback-router:apollo decides the request under its per-provider checks. X-Router-Fallback says no_servable_model, and the decision's reasons include policy_caps_exhausted. The fallback router can then serve a model the per-request check ruled out.

Check a policy before you save it

The policy preview shows which of your providers your rules allow. In the dashboard it is the Live Preview beside Organization Policies; through the API it is POST /v1/routing/policies/preview, with the policy fields you want to try (anything left out is your saved value). It answers with:

• eligible and excluded, with the reason for each exclusion;
• perRequest: the caps Artemis Task checks per request. They exclude no provider in the preview, because they depend on each prompt;
• projects: each of your projects that narrows your regions, or rules out a provider in its project policy (allowed and blocked providers, allowed regions), with its own eligible, excluded (the reason names the project's rule) and perRequest.

The preview works with a dashboard session or an organization-wide API key; a project-scoped key gets 403.

Provider order

Set it: dashboard Settings → Routing: Preferred providers under Organization Policies, and Preferred Provider Order under Configuration. API: policy.preferredProviders and providerOrder on PUT /v1/routing/config (provider_order for a project, through the API). The API also takes providerAffinity, a bonus or penalty from −1 to 1 per provider.

The two lists make one preference: your preferred providers first, in their order, then the provider order's other entries. Projects always use your organization's preferred providers, ahead of their own provider order.

Artemis Task picks among the providers that serve the chosen model: your preference first, then the lower latency, then the provider id. The preference also breaks ties between models after cost. Affinity is not used. The Fastest preset ranks each model and provider on expected time, then cost, then id.

On a request that names a model, and when the fallback router decides, the preference is a small boost and the tie-break instead: under balanced, your first provider adds up to 0.05 to the 0-to-1 score, less down the list, plus any affinity. That decides close calls but does not overturn a clear difference.

The policy preview marks your preferred providers and lists them first.

Where to set each control

Dashboard: the organization's settings are under Settings → Routing, a project's under Projects → your project → Routing. API: PUT /v1/routing/config for the organization and PUT /v1/projects/{id}/routing/config for a project. The organization's replaces the whole configuration, so read it first and send it back with your change (examples); the project's keeps every field you leave out. Changing any of these needs an owner's or admin's session: the dashboard, or in the SDKs a token from auth.login(). An API key, even an organization-wide one, can read them, and a change made with it is refused with 403.

Control	Organization: dashboard	Organization: API	Project
Strategy	Configuration, Default Strategy	strategy	Strategy; strategy
Mode, pinned provider and model	Configuration, Routing Mode and Fixed Provider & Model	mode, manualProvider, manualModel	Mode, Manual Provider, Manual Model; mode, manual_provider, manual_model
Speed	Configuration, Speed	speed	Speed (Artemis Task); router_speed ("" follows the organization)
Balanced weights	Configuration, Balanced Strategy Weights	weights	API only: weights_json ("" follows the organization)
Preferred providers	Organization Policies, Preferred providers	policy.preferredProviders	The organization's, ahead of the project's provider order
Provider order	Configuration, Preferred Provider Order	providerOrder	API only: provider_order (empty follows the organization)
Max cost, max latency	Organization Policies, Cost & Latency Limits	policy.maxCostPerRequest, policy.maxLatencyMs	The organization's
Blocked providers, provider types	Organization Policies	policy.blockedProviders, policy.allowOpenSource, policy.allowProprietary, policy.allowSelfHosted, policy.allowManaged	The organization's
Allowed regions	Organization Policies, Region Restrictions	allowedRegions (top level, not in policy)	API only: allowed_regions, which can only narrow the organization's (how regions match)
fallbackBehavior	Organization Policies, Fallback Behavior	policy.fallbackBehavior	The organization's
Allowed and blocked models and providers, regions, max tokens	None	None	Policy page; PUT /v1/projects/{id}/policy. Narrows the organization's rules for the project's requests, and no fallbackBehavior relaxes it (policy enforcement)
Monthly budget with a hard limit	None	None	Budget page; PUT /v1/projects/{id}/budget. Refuses the project's requests with 402 budget_exceeded once the month's spend reaches it (budget enforcement)

Once a project has a routing configuration of its own, its mode and strategy replace the organization's for the project's requests. Its Speed, provider order, affinity and balanced weights follow the organization's while it leaves them unset ("", an empty list, or never sent); the project's routing page sets only its mode (with the pinned provider and model), strategy, Speed and model aliases. A project that connects providers (/v1/projects/{id}/connections) uses only those: if none of them can serve a request, it is refused with 422 no_available_model rather than served by another provider. See Projects. In the SDKs: client.routing.updateConfig() / update_config() for the organization, and client.project(id).routing.setConfig() (JavaScript) / set_routing_config() (Python) for a project (JavaScript, Python).

Per request

There is no per-request strategy, weights, speed or cap, and no router to choose: Artemis Task decides every auto request (Routing). A request's provider field pins that provider (above), and naming a model skips the router.

To give different traffic different settings, use projects: send each workload with its project's API key, or name the project with the X-NuRoute-Project-ID header.

Example: "I want it cheaper"

1. Set the strategy to cheapest (Cost-first). Its quality target is 92% instead of Balanced's 97%.
2. Keep Speed at Off. Every step up costs dollars to keep the quality target.
3. Add a max cost per request if no single request may cost more than an amount. Artemis Task holds it per request. With any fallbackBehavior but fail, a prompt that every model would breach goes to the fallback router instead (above).
4. The balanced weights change nothing under Artemis Task. On a model you name, cheapest, or a higher cost weight under balanced, ranks its providers by cost.

Example: "I want faster answers"

1. Turn Speed up to Faster or Fastest. Your preset keeps its quality target and pays in dollars.
2. Or set the strategy to fastest. On a version with a speed table it runs Balanced at speed Fastest, whatever the Speed setting.
3. Add a max latency to rule out, per request, any model expected to take longer than that to finish its answer (see the caution on what the number means).
4. Time to decide. Artemis Task itself takes time on every request (a median of 127 ms on a laptop CPU, see Request handling). If the time to decide matters more to you than the time to answer, name a model: no router runs.

See also

• Routing with Artemis Task: how it decides, its presets and Speed, which router decided, fallbackBehavior
• Projects: settings per project, and project-scoped API keys
• Routing in the JavaScript SDK · in the Python SDK