Routing rules

Routing rules belong to a project and apply to every gateway request logged against it, either from a key tied to the project or one that names it in X-Dekka-Project (see AI keys). A project that has never saved rules may use every available model from both providers, with no fallback and no cap, and requests without a project follow no rules at all.

Change a project's rules

Open AI Gateway, then Routing, and pick the project. Tick the providers and models it may use, build its fallback order, set a cap if you want one and choose Save rules. The rules apply from the next request. Anyone in the organisation can see them, and developers, admins and owners can change them, which the API does with PUT /v1/projects/{project_id}/ai/policy.

Which providers and models may a project use?

A project needs at least one provider. Leave every model unticked to allow all of them from the providers you chose, or tick the ones it may use. A request for any other model answers 403 model_not_allowed, so prompts never reach a provider you haven't approved.

Saving checks that every model you name is in the catalogue, which otherwise answers 422 unknown_model, and comes from an allowed provider, which otherwise answers 422 model_provider_not_allowed. With models ticked, each model in the fallback order must be one of them, or saving answers 422 fallback_model_not_allowed.

An AI Gateway node on the canvas can narrow this further for the services connected to it. Its allowed_models, which the API sets, is a list of catalogue IDs those services may use on top of the project's rules.

How does fallback work?

When the model you asked for fails before any of its reply has reached your code, the gateway tries the next model in the project's fallback order, which holds up to 10 models. It falls back when the provider answers with a 5xx or a 429, doesn't answer in time, drops the connection, or has no key the gateway can use. If the model you asked for is in the list, the gateway carries on with the models after it, and otherwise it starts from the top. It skips any model the rules don't allow or that is no longer available.

Any other failure comes straight back, and so does a failure after a streamed reply has started. When every model fails, you get the last one's error.

Error	What happened
503 provider_unavailable	Neither you nor Dekka has a key for the provider. Add one under AI keys.
504 upstream_timeout	The provider didn't answer in time.
429 provider_rate_limited	The provider is rate-limiting the key.
502 provider_auth_failed	The provider turned the key down. Check the key you saved.
400 provider_rejected_request	The provider refused the request, and its message says why.
502 upstream_error	The provider failed in some other way.

X-Dekka-Model names the model that answered, and on the Requests page a request that fell back shows which model failed first.

Set a monthly spend cap

The cap is in US dollars at the providers' list prices, for each calendar month in UTC. Before each request the gateway compares what the project has spent this month with the cap, and once spending reaches it, the gateway answers 429 spend_cap_reached until the month ends. The error's details hold the cap_usd and the spent_usd. The Routing page shows what the project has spent so far this month.

Because the check happens before each request, the request that crosses the cap still finishes, so spending can end slightly above it. The cap is an estimate of the provider's bill at list price, and it doesn't include Dekka's own charge for AI tokens.

In what order are requests checked?

Once the key and any X-Dekka-Project header are accepted, the gateway checks each request in this order and stops at the first refusal.

1. The key's rate limit of 600 requests a minute, which answers 429 rate_limited with a Retry-After header.
2. That the model exists and is available, or 404 model_not_found.
3. That the project's rules and any AI Gateway node allow the model, or 403 model_not_allowed.
4. The project's monthly cap, or 429 spend_cap_reached.
5. Your organisation's hard spend limit, or 403 spend_limit_reached.
6. Your plan's AI token allowance, or 403 entitlement_exceeded. Free includes 100,000 AI tokens a period and refuses more, while Starter and Growth bill the tokens beyond their allowance.

Only then does the gateway call a provider. Requests refused from step 2 on appear on the Requests page as Refused, with their error code.