Providers

An AI provider is an upstream LLM service the gateway can call: one of Zuplo's built-in providers (OpenAI, Anthropic, Azure AI, Bedrock, …), which every bucket lists and nobody can change, or a custom OpenAI-compatible endpoint defined in the bucket. A provider is a catalog entry. It holds no key and routes nothing until a provider assignment uses it.

GET …/ai/providers/{providerId}/models lists the models a provider offers, with their list prices: Zuplo's catalog for a built-in provider, and the models it was defined with for a custom one.

List AI providers

GET

https://dev-api-v4-main-a471234.zuplo.app

/v4/buckets/{bucketId}/ai/providers

Lists the AI providers the gateway can use: Zuplo's built-in providers (Anthropic, OpenAI, Amazon Bedrock, …) first, ordered by type, then the custom providers defined in this bucket, oldest first. A provider is a catalog entry and holds no key: to route requests to one, create a provider assignment (POST …/ai/provider-assignments).

List AI providers › path Parameters

bucketId

string · maxLength: 64 · required

The bucket ID (bckt_…). Everything in a bucket is addressed under it.

List AI providers › query Parameters

type

ProviderType · enum

Only return providers of this type.

The kind of upstream provider. Every built-in provider has its own type; custom is a custom provider, any OpenAI-compatible endpoint. zuplo-demo is Zuplo's demo provider. New values may be added.

Enum values:

anthropic

azure-ai

bedrock-mantle

bedrock-runtime

custom

google

mistral

moonshot

limit

integer · min: 1 · max: 100

The maximum number of items to return.

Default: 10

offset

integer · min: 0

The number of items to skip before the first item returned.

Default: 0

List AI providers › Responses

A page of AI providers.

AiProviderList

AiProvider[] · required

The providers on this page.

limit

integer · required

The page size that was applied.

offset

integer · required

The number of items skipped before this page.

total

integer · required

The number of items in the whole collection.

hasMore

boolean · required

Whether there are more items after this page.

next

string | null · required

The relative URL of the next page, or null on the last page.

Create a custom AI provider

POST

https://dev-api-v4-main-a471234.zuplo.app

/v4/buckets/{bucketId}/ai/providers

Defines a custom provider: an OpenAI-compatible endpoint that is not one of Zuplo's built-in providers, and the models it serves. Its type is custom. A provider holds no key and routes nothing on its own: assign it to the gateway with POST …/ai/provider-assignments, which names the environment variable that holds its key.

name must be unique among the bucket's custom providers. Each model needs its kind and pricing, because a custom provider's models are not in Zuplo's catalog. Built-in providers cannot be created.

Create a custom AI provider › path Parameters

bucketId

string · maxLength: 64 · required

The bucket ID (bckt_…). Everything in a bucket is addressed under it.

Create a custom AI provider › Request Body

A new custom AI provider. Its `type` is `custom`.

CreateAiProviderRequest

name

string · minLength: 1 · maxLength: 128 · required

The provider's display name, unique among the bucket's custom providers.

baseUrl

string · uri · maxLength: 2048 · required

The OpenAI-compatible endpoint.

AiCustomModelInput[] · minItems: 1 · maxItems: 100 · required

The models the provider offers, each with its kind and prices.

description

string | null · maxLength: 1024

A note for people managing the gateway.

Create a custom AI provider › Responses

The custom provider was created.

An AI provider the gateway can use: one of Zuplo's built-in providers, or a custom provider defined in the bucket. It holds no key and routes nothing on its own; a provider assignment does.

AiProvider

id

string · required

The provider ID: a built-in provider's name (Anthropic, BedrockRuntime, …), or a custom provider's ID (aiprov_…).

Example: Anthropic

bucketId

string | null · required

The bucket that defines the custom provider. Null for a built-in provider.

Example: bckt_Y40ypGODCDP3S7iZxNwCAbIe

type

ProviderType · enum · required

The kind of upstream provider. Every built-in provider has its own type; custom is a custom provider, any OpenAI-compatible endpoint. zuplo-demo is Zuplo's demo provider. New values may be added.

Enum values:

anthropic

azure-ai

bedrock-mantle

bedrock-runtime

custom

google

mistral

moonshot

builtIn

boolean · required

Whether this is one of Zuplo's built-in providers, which are read-only. False for a custom provider.

name

string · required

The provider's display name. It is not a routing prefix; an assignment's label is.

Example: Anthropic

description

string | null · required

A note for people managing the gateway.

baseUrl

string | null · uri · required

The upstream endpoint. Null for a built-in provider whose endpoint is set per assignment (bedrock-mantle, azure-ai, vertex-ai).

Example: https://api.anthropic.com

createdOn

string | null · date-time · required

When the custom provider was defined. Null for a built-in provider.

updatedOn

string | null · date-time · required

When the custom provider was last changed. Null for a built-in provider.

POST/v4/buckets/{bucketId}/ai/providers

curl https://dev-api-v4-main-a471234.zuplo.app/v4/buckets/:bucketId/ai/providers \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <token>' \
  --data '{
  "name": "Llama on our cluster",
  "description": "Llama 3 on our inference cluster",
  "baseUrl": "https://llm.internal.example.com/v1",
  "models": [
    {
      "name": "llama-3.1-70b",
      "kind": "completions",
      "pricing": {
        "inputPerMillionTokens": 0.2,
        "outputPerMillionTokens": 0.6
      }
    }
  ]
}'

Example Request Body

{
  "name": "Llama on our cluster",
  "description": "Llama 3 on our inference cluster",
  "baseUrl": "https://llm.internal.example.com/v1",
  "models": [
    {
      "name": "llama-3.1-70b",
      "kind": "completions",
      "pricing": {
        "inputPerMillionTokens": 0.2,
        "outputPerMillionTokens": 0.6
      }
    }
  ]
}

json

application/json

Example Responses

{
  "id": "Anthropic",
  "bucketId": "bckt_Y40ypGODCDP3S7iZxNwCAbIe",
  "type": "anthropic",
  "builtIn": true,
  "name": "Anthropic",
  "description": "description",
  "baseUrl": "https://api.anthropic.com",
  "createdOn": "2024-08-25T15:00:00Z",
  "updatedOn": "2024-08-25T15:00:00Z"
}

json

application/json

Get an AI provider

GET

https://dev-api-v4-main-a471234.zuplo.app

/v4/buckets/{bucketId}/ai/providers/{providerId}

Returns a built-in or custom AI provider. Its models, with their prices, are at GET …/ai/providers/{providerId}/models.

Get an AI provider › path Parameters

bucketId

string · maxLength: 64 · required

The bucket ID (bckt_…). Everything in a bucket is addressed under it.

providerId

string · maxLength: 64 · required

The AI provider ID, a built-in provider's name (Anthropic) or a custom provider's ID (aiprov_…).

Get an AI provider › Responses

The AI provider.

An AI provider the gateway can use: one of Zuplo's built-in providers, or a custom provider defined in the bucket. It holds no key and routes nothing on its own; a provider assignment does.

AiProvider

id

string · required

The provider ID: a built-in provider's name (Anthropic, BedrockRuntime, …), or a custom provider's ID (aiprov_…).

Example: Anthropic

bucketId

string | null · required

The bucket that defines the custom provider. Null for a built-in provider.

Example: bckt_Y40ypGODCDP3S7iZxNwCAbIe

type

ProviderType · enum · required

The kind of upstream provider. Every built-in provider has its own type; custom is a custom provider, any OpenAI-compatible endpoint. zuplo-demo is Zuplo's demo provider. New values may be added.

Enum values:

anthropic

azure-ai

bedrock-mantle

bedrock-runtime

custom

google

mistral

moonshot

builtIn

boolean · required

Whether this is one of Zuplo's built-in providers, which are read-only. False for a custom provider.

name

string · required

The provider's display name. It is not a routing prefix; an assignment's label is.

Example: Anthropic

description

string | null · required

A note for people managing the gateway.

baseUrl

string | null · uri · required

The upstream endpoint. Null for a built-in provider whose endpoint is set per assignment (bedrock-mantle, azure-ai, vertex-ai).

Example: https://api.anthropic.com

createdOn

string | null · date-time · required

When the custom provider was defined. Null for a built-in provider.

updatedOn

string | null · date-time · required

When the custom provider was last changed. Null for a built-in provider.

Update a custom AI provider

PATCH

https://dev-api-v4-main-a471234.zuplo.app

/v4/buckets/{bucketId}/ai/providers/{providerId}

Updates a custom provider with JSON Merge Patch semantics. Omitted properties are unchanged and null clears a property. Built-in providers are read-only: changing one fails with 409.

models replaces all of the provider's models, so send the complete list. Removing a model fails with 409 while a provider assignment still selects it. When a provider assignment uses the provider, a change deploys the gateway: it creates a zuplo deployment of the production environment. Read the newest one until it has succeeded; that does not guarantee every edge serves the change yet.

Update a custom AI provider › path Parameters

bucketId

string · maxLength: 64 · required

The bucket ID (bckt_…). Everything in a bucket is addressed under it.

providerId

string · maxLength: 64 · required

The AI provider ID, a built-in provider's name (Anthropic) or a custom provider's ID (aiprov_…).

Update a custom AI provider › Request Body

Changes to a custom AI provider (JSON Merge Patch).

UpdateAiProviderRequest

name

string · minLength: 1 · maxLength: 128

The provider's display name, unique among the bucket's custom providers.

description

string | null · maxLength: 1024

A note for people managing the gateway. Null clears it.

baseUrl

string · uri · maxLength: 2048

The OpenAI-compatible endpoint.

AiCustomModelInput[] · minItems: 1 · maxItems: 100

Replaces all of the models the provider offers. Send the complete list.

Update a custom AI provider › Responses

The updated custom provider.

An AI provider the gateway can use: one of Zuplo's built-in providers, or a custom provider defined in the bucket. It holds no key and routes nothing on its own; a provider assignment does.

AiProvider

id

string · required

The provider ID: a built-in provider's name (Anthropic, BedrockRuntime, …), or a custom provider's ID (aiprov_…).

Example: Anthropic

bucketId

string | null · required

The bucket that defines the custom provider. Null for a built-in provider.

Example: bckt_Y40ypGODCDP3S7iZxNwCAbIe

type

ProviderType · enum · required

The kind of upstream provider. Every built-in provider has its own type; custom is a custom provider, any OpenAI-compatible endpoint. zuplo-demo is Zuplo's demo provider. New values may be added.

Enum values:

anthropic

azure-ai

bedrock-mantle

bedrock-runtime

custom

google

mistral

moonshot

builtIn

boolean · required

Whether this is one of Zuplo's built-in providers, which are read-only. False for a custom provider.

name

string · required

The provider's display name. It is not a routing prefix; an assignment's label is.

Example: Anthropic

description

string | null · required

A note for people managing the gateway.

baseUrl

string | null · uri · required

The upstream endpoint. Null for a built-in provider whose endpoint is set per assignment (bedrock-mantle, azure-ai, vertex-ai).

Example: https://api.anthropic.com

createdOn

string | null · date-time · required

When the custom provider was defined. Null for a built-in provider.

updatedOn

string | null · date-time · required

When the custom provider was last changed. Null for a built-in provider.

PATCH/v4/buckets/{bucketId}/ai/providers/{providerId}

curl https://dev-api-v4-main-a471234.zuplo.app/v4/buckets/:bucketId/ai/providers/:providerId \
  --request PATCH \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <token>' \
  --data '{
  "models": [
    {
      "name": "llama-3.1-70b",
      "kind": "completions",
      "pricing": {
        "inputPerMillionTokens": 0.2,
        "outputPerMillionTokens": 0.6
      }
    },
    {
      "name": "bge-large-en",
      "kind": "embeddings",
      "pricing": {
        "inputPerMillionTokens": 0.02,
        "outputPerMillionTokens": 0
      }
    }
  ]
}'

Example Request Body

{
  "models": [
    {
      "name": "llama-3.1-70b",
      "kind": "completions",
      "pricing": {
        "inputPerMillionTokens": 0.2,
        "outputPerMillionTokens": 0.6
      }
    },
    {
      "name": "bge-large-en",
      "kind": "embeddings",
      "pricing": {
        "inputPerMillionTokens": 0.02,
        "outputPerMillionTokens": 0
      }
    }
  ]
}

json

application/json

Example Responses

{
  "id": "Anthropic",
  "bucketId": "bckt_Y40ypGODCDP3S7iZxNwCAbIe",
  "type": "anthropic",
  "builtIn": true,
  "name": "Anthropic",
  "description": "description",
  "baseUrl": "https://api.anthropic.com",
  "createdOn": "2024-08-25T15:00:00Z",
  "updatedOn": "2024-08-25T15:00:00Z"
}

json

application/json

Delete a custom AI provider

DELETE

https://dev-api-v4-main-a471234.zuplo.app

/v4/buckets/{bucketId}/ai/providers/{providerId}

Deletes a custom provider and its models. Fails with 409 while a provider assignment uses it: delete the assignment first. Built-in providers are read-only: deleting one fails with 409. A repeated delete returns 404, which clients should treat as success.

Delete a custom AI provider › path Parameters

bucketId

string · maxLength: 64 · required

The bucket ID (bckt_…). Everything in a bucket is addressed under it.

providerId

string · maxLength: 64 · required

The AI provider ID, a built-in provider's name (Anthropic) or a custom provider's ID (aiprov_…).

Delete a custom AI provider › Responses

The custom provider was deleted.

No data returned

List a provider's models

GET

https://dev-api-v4-main-a471234.zuplo.app

/v4/buckets/{bucketId}/ai/providers/{providerId}/models

Lists the models a provider offers, with their list prices, ordered by kind and name. For a built-in provider, they are the models Zuplo's catalog knows for it; for a custom provider, the models it was defined with. A provider assignment selects some of them by name in its models.

List a provider's models › path Parameters

bucketId

string · maxLength: 64 · required

The bucket ID (bckt_…). Everything in a bucket is addressed under it.

providerId

string · maxLength: 64 · required

The AI provider ID, a built-in provider's name (Anthropic) or a custom provider's ID (aiprov_…).

List a provider's models › query Parameters

kind

ModelKind · enum

Only return models of this kind.

What a model is used for. New values may be added.

Enum values:

completions

embeddings

limit

integer · min: 1 · max: 100

The maximum number of items to return.

Default: 10

offset

integer · min: 0

The number of items to skip before the first item returned.

Default: 0

List a provider's models › Responses

A page of the provider's models.

A page of a provider's models.

AiProviderModelList

AiProviderModel[] · required

The models on this page.

limit

integer · required

The page size that was applied.

offset

integer · required

The number of items skipped before this page.

total

integer · required

The number of items in the whole collection.

hasMore

boolean · required

Whether there are more items after this page.

next

string | null · required

The relative URL of the next page, or null on the last page.

GET/v4/buckets/{bucketId}/ai/providers/{providerId}/models

curl https://dev-api-v4-main-a471234.zuplo.app/v4/buckets/:bucketId/ai/providers/:providerId/models \
  --header 'Authorization: Bearer <token>'

Example Responses

{
  "data": [
    {
      "id": "Rz3Kp8VnQ1wT6yLm4sHb9XcF",
      "providerId": "Anthropic",
      "name": "claude-sonnet-4-6",
      "kind": "completions",
      "status": "active",
      "pricing": {
        "currency": "usd",
        "inputPerMillionTokens": 0.15,
        "outputPerMillionTokens": 0.6,
        "cachedInputPerMillionTokens": 0,
        "cacheWritePerMillionTokens": 0,
        "cacheWrite1hPerMillionTokens": 0
      }
    }
  ],
  "limit": 0,
  "offset": 0,
  "total": 0,
  "hasMore": true,
  "next": "next"
}

json

application/json

Get a provider's model

GET

https://dev-api-v4-main-a471234.zuplo.app

/v4/buckets/{bucketId}/ai/providers/{providerId}/models/{modelId}

Returns one model a provider offers, with its list prices.

Get a provider's model › path Parameters

bucketId

string · maxLength: 64 · required

The bucket ID (bckt_…). Everything in a bucket is addressed under it.

providerId

string · maxLength: 64 · required

The AI provider ID, a built-in provider's name (Anthropic) or a custom provider's ID (aiprov_…).

modelId

string · maxLength: 64 · required

The ID of a model the provider offers.

Get a provider's model › Responses

The model.

A model a provider offers, and its list price.

AiProviderModel

id

string · required

The model ID.

Example: Rz3Kp8VnQ1wT6yLm4sHb9XcF

providerId

string · required

The provider that offers the model.

Example: Anthropic

name

string · required

The model's name at the provider, unique per provider. A provider assignment selects the model by this name.

Example: claude-sonnet-4-6

kind

ModelKind · enum · required

What a model is used for. New values may be added.

Enum values:

completions

embeddings

status

ModelStatus · enum · required

Whether a model is current or on its way out. New values may be added.

Enum values:

active

deprecated

AiModelPricing · required

Prices in US dollars per one million tokens. They drive cost metering and cost budgets. Null means the token type is not priced.

Budget Rules Provider Assignments