LLM gateways

Manage model access, track usage, and set budgets for agents and users.

An LLM gateway runs in your cluster and connects agents to model providers. Open Capabilities → Gateways.

What a gateway does

Manage keys: keep provider credentials in the gateway and give agents short-lived virtual keys.
Track usage: requests, tokens, cache reads and writes, and spend by agent and user.
Set limits: session and period budgets, plus request and token rates for each agent or user.

Create a gateway

A gateway's Overview tab with health, synced providers, and the access panel — The Overview tab: health, synced provider keys, and who may use the gateway.

Choose a deployment and DNS-style name. Optionally make it the deployment's default; only one is allowed. Status moves from pending to provisioning to active. The detail page shows the endpoint and any error.

The Create Gateway dialog with name and namespace fields, a Set as default gateway toggle and Shared or Private visibility — Set the name, namespace, default status, and access.

The Gateways list showing one active default gateway with its deployment and namespace — A star marks the default gateway. Filter the list by status.

Sync providers into it

Choose which providers to sync. Sync copies their credentials into the gateway so agents and virtual keys can use them. Later edits sync automatically. Removing a provider removes its key from the gateway.

Check the link status

Each provider link shows its last sync and any error. Check it if model requests fail even though the gateway is active.

Route agents through it

Set Use gateway when creating or editing agents, apps, or web agents:

Setting	Effect
A named gateway	Always use this gateway.
Deployment default	Preselected for new agents. Existing agents keep their saved gateway until edited.
Off	Use provider keys directly.

agent.yaml· yaml

1spec:
2  gateway:
3    name: shared-gateway
4    namespace: adaptive

Changing an agent's gateway needs set-agent-gateway support and takes effect on the next pod.

Virtual keys

Industric creates a virtual key for each session, agent, and provider. It is shown once and expires when the session ends. Revoke it anytime from the key list. The gateway tracks each key's spend, requests, and input, output, and cached tokens.

Administrators can also create keys for applications or testing. User keys must be limited to a named provider.

using a minted key· bash

export OPENAI_BASE_URL=https://<gateway-endpoint>/v1

export OPENAI_API_KEY=<minted-virtual-key>

Budgets & rate limits

Target named agents or users, or * for all. Limits apply to each target separately:

Field	Caps
`session_budget_usd`	Spend on a single minted key, i.e. one run or session.
`total_budget_usd`	Spend across all of the target's keys until the reset boundary.
`resets`	When the total resets: `never`, `daily`, `weekly`, or `monthly`.
`requests_per_minute`	Request rate. Absent or zero means unlimited.
`tokens_per_minute`	Token rate. Absent or zero means unlimited.

Usage updates can lag

Usage takes time to sync across gateway replicas, so spend can exceed a cap. Allow headroom when setting budgets, especially with more replicas.

Usage analytics

The Analytics tab charting gateway spend with the cache read/write breakdown — The Analytics tab: spend, requests, and tokens per gateway, with cached tokens broken out.

In Usage, choose a time range and view spend, requests, and tokens by agent, user, or day. Cache spend is shown separately. Filter by agent or user for details.

Scaling

Edit replica CPU and memory settings on the detail page to restart the gateway with new limits. Clear the fields to restore defaults. Deleting a gateway reports any credentials that still need cleanup.

llmgateway.yaml· yaml

1apiVersion: autonomic.sh/v1alpha1
2kind: LLMGateway
3metadata:
4  name: shared-gateway
5  namespace: adaptive
6spec:
7  replicas: 2
8  service:
9    type: ClusterIP
10    port: 4000
11  resources:
12    requests:
13      cpu: 500m
14      memory: 1Gi

Managed by the dashboard

Configure providers, pricing, and budgets in the dashboard. Industric syncs them to the gateway; do not add them to the resource YAML.