How to choose an LLM for each task automatically

An application can receive a short translation request, a long extraction task and a difficult analysis question through the same text box. Sending them to one model is straightforward. Choosing different models automatically takes a little more design.

You need an eligible model pool, a selection rule and evidence that the resulting answers meet your requirements. Adding auto to a request is the interface to that system, not proof that the system is choosing well.

Choose the unit of work

Decide whether you are routing individual requests, complete conversations or whole agent tasks. These are not interchangeable.

A request-level router may choose a different model on the next turn. That can affect style, tool behavior and provider caching. If your product is conversational, evaluate the conversation rather than a collection of unrelated opening prompts.

Set eligibility before preferences

List the hard requirements: approved providers, allowed regions, input size, output needs and tool support. Remove candidates that cannot serve the request before comparing quality or cost.

Then set the preference: lower cost, higher expected quality or faster completion. Keep a failure path for the case where no candidate meets the requirements. Do not silently broaden a strict policy to make the request succeed.

Know what drives the selection

Simple rules work when you already know the right destination. A classifier can estimate the task category. Learned routers can predict relative performance from training data. The appropriate mechanism depends on what the system needs to infer.

Artemis profiles the task mix, maps it to expected model scores, estimates answer length and cost, and selects according to the preset. Its non-generative predictors run in the routing path. The expected scores are estimates, not grades of an answer that already exists. Artemis selection method

Send a minimal request through NuRoute

For the Platform API, use the gateway URL supplied by your deployment and a Platform API key. Set NUROUTE_GATEWAY_URL to the gateway origin without a trailing slash or /v1. Set NUROUTE_API_KEY to your key in your local environment.

curl --fail-with-body "$NUROUTE_GATEWAY_URL/v1/chat/completions" \
  -H "Authorization: Bearer $NUROUTE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      {"role": "user", "content": "Summarise this release note in one sentence: The export now preserves column order."}
    ]
  }'

This request shape follows the production quickstart. It is a documentation-based example, not an authenticated test run. NuRoute Chat's separate API uses artemis; do not mix its key or model name with the Platform API setup.

Inspect what actually happened

Check the response's model field and the routing headers. X-Router-Version: v2 identifies Artemis; v1 indicates that the fallback router made the decision. X-Router-Fallback gives the fallback reason when present. The decision trace provides more context. Routing diagnostics

Logging only the requested alias auto hides the result you need to evaluate. Record the served model, router version, capability version, usage, errors and completion time.

Check the current constraints

NuRoute currently ignores response_format. Validate structured output in your application, and do not assume a JSON instruction guarantees a schema. The provider-level tool check also does not establish model-level tool reliability. For a critical tool workflow, evaluate a fixed model through an Enterprise gateway or another provider first; standard NuRoute is Artemis-only. Request handling

For automatic Cloud requests, the documented Quality-first setting currently resolves to Balanced. Preset quality targets come from training data and are not guarantees about your workload. Self-hosted NuRoute uses its fallback router rather than Artemis. Preset availability

Validate the choice with held-out work

Compare the router with a capable fixed model and an inexpensive fixed model. Grade answers without model names, inspect failures by task type and check full task cost. Where predictability requires a named-model override, use an Enterprise gateway or a separate provider integration. Standard NuRoute does not expose individual-model selection.

Follow the benchmark guide for a repeatable evaluation, or the n8n workflow guide for a visual setup.