Create a routing config

Authorization

• AuthorizationBearer va_

  A platform API key (va_...), created under Settings > API keys. Scoped to an organization or one workspace, with an admin or member role. Used by the management API.

Request body application/json

• namestringrequired

  The config's name, unique within the organization. The slug is derived from it. Required.

• organizationstringrequired

  The slug of the organization the config belongs to. Required.

• workspace_idstring<uuid>required

  The workspace the config belongs to (UUID). Required.

• strategy_modeenum<string>required

  How a request picks among targets. Required.

  • SINGLE: always the first target.
  • LOADBALANCE: one target at random, in proportion to each target's weight.
  • FALLBACK: the targets in order; the next is tried when one fails with a status in on_status_codes.
  • CONDITIONAL: the first of conditions (by condition_order) that matches the request picks its then_target_name; if none matches, conditional_default_target_name.

  One of "SINGLE", "LOADBALANCE", "FALLBACK", "CONDITIONAL"

• targetsobject[]required

  The targets, in order. At least one. Required.

  Child attributes

  • namestring

    The target's name, which conditions and conditional_default_target_name refer to. Optional otherwise.

  • virtual_key_slugstring

    For a virtual key target: the key's slug. The key must belong to the config's organization and not be EXPIRED. Requests to this target use the key's provider credential.

  • weightnumber<double>

    For LOADBALANCE: the target's share of requests, relative to its siblings. Greater than 0. Default 1.

  • order_indexinteger<int32>

    The target's position among its siblings, which sets the FALLBACK order and which target SINGLE uses. Absent: its position in the list.

  • strategy_modeenum<string>

    For a nested strategy target: how it picks among its own targets: SINGLE, LOADBALANCE, FALLBACK or CONDITIONAL.

    One of "SINGLE", "LOADBALANCE", "FALLBACK", "CONDITIONAL"

  • on_status_codesinteger<int32>[]

    For a nested FALLBACK target: the provider HTTP statuses that move on to its next target. Empty or absent: 429, 500, 502, 503 and 504.

  • conditional_default_target_namestring

    For a nested CONDITIONAL target: the name of one of its targets, used when no condition matches.

  • conditionsobject[]

    For a nested CONDITIONAL target (at least one required): the rules that pick one of its targets.

    Child attributes

    • condition_orderinteger<int32>required

      The rule's position: lower numbers are tried first. Required.

    • querymap<string, object>required

      What the request must match, as a JSON object of field paths to operator objects, all of which must hold, e.g. {"metadata.tier": {"$eq": "premium"}}. Required and not empty.

      • Fields: metadata. followed by a key of the JSON object in the request's x-metadata header, and params.model, params.temperature, params.max_tokens, params.top_p, params.stream (an omitted stream is false).
      • Operators: $eq, $ne, $gt, $gte, $lt, $lte (numbers), $in, $nin (arrays), $regex (case-insensitive, must match the whole value) and $exists (true or false).
      • $and and $or take an array of such objects.
    • then_target_namestringrequired

      The name of the target, at the same level as this rule, that a matching request goes to. Required.

  • override_paramsobject

    Request parameters a routing target sets on the requests it receives, in place of the caller's. A field left out keeps the caller's value. model, temperature, max_tokens, top_p and stop are applied; the other fields are validated and stored but not applied at present.

    Child attributes

    • modelstring

      The model to call instead of the request's, e.g. gpt-4o-mini.

    • temperaturenumber<double>

      The sampling temperature, from 0 to 2.

    • max_tokensinteger<int32>

      The most tokens to generate; at least 1.

    • top_pnumber<double>

      Nucleus sampling, from 0 to 1.

    • frequency_penaltynumber<double>

      From -2 to 2. Stored, not applied at present.

    • presence_penaltynumber<double>

      From -2 to 2. Stored, not applied at present.

    • stopstring[]

      Stop sequences; a non-empty list replaces the request's.

    • seedinteger<int32>

      A sampling seed, at least 0. Stored, not applied at present.

    • response_format_typeenum<string>

      json_object, text or json_schema. Accepted, but neither applied nor returned at present.

      One of "json_object", "text", "json_schema"

  • cacheobject

    Response cache settings stored with a routing config or target. Requests served through a routing config are not answered from the response cache at present.

    Child attributes

    • modestring

      The cache mode: simple (exact match) or semantic. Absent: none.

    • max_age_secondsinteger<int32>

      How long a cached answer stays valid, in seconds. Absent: none.

  • retryobject

    Retries for a routing target's provider calls. A failed call with a listed status is tried again after 1, 2, 4, ... seconds (at most 16), until attempts calls have been made.

    Child attributes

    • attemptsinteger<int32>required

      How many calls to make in all, the first one included (1 means no retry). From 1 to 10. Required.

    • on_status_codesinteger<int32>[]

      The provider HTTP statuses to retry. Empty or absent: 429, 500, 502, 503 and 504.

  • request_timeout_msinteger<int32>

    For a virtual key target: the time limit for each call to its provider, in milliseconds. A call that takes longer fails. Absent: no limit of its own.

• descriptionstring

  A free-text description. Optional.

• on_status_codesinteger<int32>[]

  For FALLBACK: the provider HTTP statuses that move a request on to the next target. Any other failure is returned at once. Empty or absent: 429, 500, 502, 503 and 504.

• conditional_default_target_namestring

  For CONDITIONAL: the name of the target used when no condition matches. Absent: a request that matches no condition fails with 424.

• conditionsobject[]

  For CONDITIONAL (at least one required): the rules that pick a target. Ignored otherwise.

  Child attributes

  • condition_orderinteger<int32>required

    The rule's position: lower numbers are tried first. Required.

  • querymap<string, object>required

    What the request must match, as a JSON object of field paths to operator objects, all of which must hold, e.g. {"metadata.tier": {"$eq": "premium"}}. Required and not empty.

    • Fields: metadata. followed by a key of the JSON object in the request's x-metadata header, and params.model, params.temperature, params.max_tokens, params.top_p, params.stream (an omitted stream is false).
    • Operators: $eq, $ne, $gt, $gte, $lt, $lte (numbers), $in, $nin (arrays), $regex (case-insensitive, must match the whole value) and $exists (true or false).
    • $and and $or take an array of such objects.
  • then_target_namestringrequired

    The name of the target, at the same level as this rule, that a matching request goes to. Required.

• cacheobject

  Response cache settings stored with a routing config or target. Requests served through a routing config are not answered from the response cache at present.

  Child attributes

  • modestring

    The cache mode: simple (exact match) or semantic. Absent: none.

  • max_age_secondsinteger<int32>

    How long a cached answer stays valid, in seconds. Absent: none.

• retryobject

  Retries for a routing target's provider calls. A failed call with a listed status is tried again after 1, 2, 4, ... seconds (at most 16), until attempts calls have been made.

  Child attributes

  • attemptsinteger<int32>required

    How many calls to make in all, the first one included (1 means no retry). From 1 to 10. Required.

  • on_status_codesinteger<int32>[]

    The provider HTTP statuses to retry. Empty or absent: 429, 500, 502, 503 and 504.

• request_timeout_msinteger<int32>

  A time limit in milliseconds. Stored with the config; only the request_timeout_ms of a virtual key target is applied to its calls. Optional.

Responses