Copy page

AI Gateway

AI Gateway Resources

The AI Gateway is the first part of Zuplo exposed through v4. Today it is configured through api.zuplo.com with one generic configuration resource, an untyped metadata blob, and URLs addressed by bucket. v4 replaces that with typed resources that match how people think about the product.

The resource model

Code
account (by name)
├── project (type ai-gateway)                          /v4/projects/{projectId}
└── bucket (the project's production bucket)           /v4/buckets/{bucketId}
    ├── budget rules   all of the gateway's traffic    …/ai/budget-rules
    ├── AI providers   built-in (read-only) + custom   …/ai/providers/{providerId}
    │   └── models     list prices                     …/ai/providers/{providerId}/models
    ├── AI provider assignments  label, models, key    …/ai/provider-assignments/{assignmentId}
    ├── AI pools       budget, policy template, people …/ai/pools/{poolId}
    │   ├── members    (user, role)                    …/ai/pools/{poolId}/members/{userId}
    │   ├── sub-pools  (pools with parentPoolId)
    │   └── AI apps    one client each                 …/ai/apps/{appId}
    │       └── consumers  one per client or env.      in the app; POST …/ai/apps/{appId}/consumers
    │           └── API keys  current + rolled         …/consumers/{consumerId}/api-keys
    └── AI user apps   personal keys for everyone      …/ai/user-apps/{userAppId}
        └── people     everyone who can use it         …/ai/user-apps/{userAppId}/users/{userId}
            ├── consumers  e.g. one per laptop         …/users/{userId}/consumers
            │   └── API keys  current + rolled         …/consumers/{consumerId}/api-keys
            ├── budget  their own limits               …/users/{userId}/budget
            └── usage                                  …/users/{userId}/usage

… is /v4/buckets/{bucketId}, the prefix of everything in the bucket. Every resource with an ID is addressed directly in the bucket (re-rooting). The tree shows ownership, not URL depth.

Resources

Projects. An AI Gateway is a project of type ai-gateway. Creating one creates its production environment and bucket, and deploys the gateway without a Git repository. The project lives in the account, and its buckets list the bucket of each environment: one, the production bucket, for an AI Gateway. Everything the gateway holds is in that bucket.

Budget rules. The gateway's own budget, for all of its traffic and on top of the budgets of its pools and apps: a singleton with no ID, …/ai/budget-rules, whose body is the list of rules, read with a GET and replaced with a PUT of the whole list. The gateway's URL, and whether it serves a change yet, are the project's production environment and its deployments.

AI providers. An upstream LLM service the gateway can call, and the models it offers. Every bucket lists Zuplo's built-in providers (Anthropic, OpenAI, Amazon Bedrock, …), which are read-only, and the custom providers defined in it: an OpenAI-compatible endpoint with its own models, each with a kind and pricing. A provider holds no key and routes nothing on its own. Its models, with their list prices in US dollars per million tokens, are at …/ai/providers/{providerId}/models.

AI provider assignments. The gateway's choice to use a provider: a routing label, the provider's models it serves, and the name of the environment variable that holds the provider's API key. Clients select a model in requests as <label>/<model> (anthropic1/claude-sonnet-4-6), and routing policies name it the same way, the model's ref. So a label cannot be changed yet. The key is never sent to the API: the gateway reads the variable from the bucket's project when it calls the provider, so a missing variable fails requests, not the write. A model can have negotiated prices, which replace its list prices in cost metering and budgets. One provider can be assigned several times, with different labels, models or keys.

Policies. Everything an app does to a request is a policy, run in order: authentication (auth, or internal-only for apps that serve other policies), model routing (model-filtering, model-override, fallback-model, load-balancing, smart-router), budgets (metering, whose budgetRules count usage per app or per person), guardrails (prompt-injection, dlp, akamai-firewall), caching (semantic-cache) and tracing (opik-tracing, galileo-tracing). A custom policy runs a policy declared in the project. Each policy is a typed variant of one union, chosen by its type.

AI pools (formerly "teams"). A pool groups apps that share:

• budgetRules, the pool's own budget, for the combined traffic of its apps and sub-pools;
• policyTemplate, the policies a new app starts with, and whether apps may change or remove each one. Sub-pools use their parent's template unless they have their own;
• people, through pool members (admin or member, inherited by sub-pools).

AI apps. One client of the gateway (a product feature, a service, an agent), in exactly one pool. It has:

• policies: a copy of the pool's template when it is created, then its own;
• baseUrl: where its clients send OpenAI- and Anthropic-compatible requests;
• consumers, one per client or environment, each with its API keys, created separately. A key's secret is shown once.

AI user apps. Personal API keys for everyone in the account, for coding agents (Claude Code, Cursor), scripts and notebooks. It has policies like an app. Its metering policy's per-person rule sets the limits every person gets. Everything about one person is under …/users/{userId}: their consumers, their budget, which replaces those limits for them, and their usage. …/users lists everyone who can use the user app, each with their budget when you ask for it with include[]=budget. Every gateway has exactly one user app, created with the gateway: it is read and changed, never created or deleted.

Consumers and API keys. A consumer is one credential, and its API keys are its versions. An app holds a consumer for each client or environment, and a person in the user app holds their own, for example one per laptop. An app's consumers are created in the bucket (…/consumers) and linked to the app by their ID (POST …/ai/apps/{appId}/consumers with the consumerId). A person's are created under the person (…/ai/user-apps/{userAppId}/users/{userId}/consumers). Both are addressed by their ID (…/consumers/{consumerId}, …/api-keys/{apiKeyId}). An app returns its consumers in consumers with include[]=consumers, and a person's are listed at their consumers. Neither names its app or person. A POST to …/consumers/{consumerId}/api-keys rolls: the consumer's other keys expire right away or after the previousKeyGracePeriodSeconds in the query. People manage their own consumers, and the user app's admins can do the same for anyone, one request per person. A key's secret is shown once, when it is created, except to the person who holds a personal consumer, who can read it again.

Typed configuration instead of metadata

An app's configuration is still the policy chain the gateway runs, but each policy is typed instead of a name and untyped options inside metadata:

Code
// today: GET /v1/buckets/{bucketId}/configurations/{configurationId}
{
  "metadata": {
    "type": "ai-app",
    "version": "2",
    "deploymentName": "integration-tests-v2-main-9f628be",
    "inboundPolicyChain": [
      {
        "name": "ai-gateway-model-filtering-inbound",
        "options": {
          "models": {
            "completions": {
              "allowList": ["openai/gpt-4o-mini", "openai/gpt-5-nano"]
            }
          }
        }
      },
      {
        "name": "ai-gateway-metering-inbound",
        "options": {
          "budgetRules": [
            {
              "budgetBy": "app",
              "meters": [
                {
                  "action": "block",
                  "meter": "requests",
                  "period": "daily",
                  "value": 100000
                }
              ]
            }
          ]
        }
      }
    ]
  }
}

Code
// v4: GET /v4/buckets/{bucketId}/ai/apps/{appId}
{
  "policies": [
    {
      "type": "model-filtering",
      "enabled": true,
      "completions": {
        "allowList": ["openai1/gpt-4o-mini", "openai1/gpt-5-nano"],
        "blockList": null
      },
      "embeddings": null
    },
    {
      "type": "metering",
      "enabled": true,
      "budgetRules": [
        {
          "budgetBy": "app",
          "expression": null,
          "limits": [
            {
              "meter": "requests",
              "period": "daily",
              "action": "block",
              "value": 100000
            }
          ]
        }
      ],
      "failOnError": false
    }
  ]
}

Every field is in the schema, so it is validated at write time (400 with every invalid field listed), documented in the API reference, and available to MCP tools and Terraform. Internal details (deploymentName, version, the names policies are declared under in policies.json, $schema, consumers) no longer leak.

See the API reference for every operation, the quickstart for the shortest path, and the end-to-end tutorial for every feature, starting from a new project.