← All engineering notes

AI product engineering

Manage approved models for each enterprise customer

Keep a customer-specific model catalog that survives provider changes, fallback and a security review.

An approved-model list is a customer contract, not a UI preference. It answers which model identifiers, providers and processing routes may handle one tenant's request. Make the list explicit before a request reaches routing. A fallback that chooses an unapproved model has broken the contract, even if it returns a better answer.

Store an eligibility record, not a list of display names

Provider catalogues change. Anthropic distinguishes active, legacy, deprecated and retired models, and retired model requests fail. Its deprecation guide also gives recommended replacements and retirement dates. Treat a provider's list as input to your review process, not as the customer’s permission.

For each customer and workload, record the exact model ID, provider route, processing region, allowed features, approval owner, review date and replacement plan. The OpenAI Models API can report model objects available to an API key, but availability does not establish that a customer approved the model.

Field Example Decision it supports
Workload support-answering Stops an experimental model from serving a production feature
Model and route gpt-5-mini, regional OpenAI connection Checks both identity and destination
Feature requirements text, streaming, tool calls Avoids a fallback that changes application behavior
Validity approved until 2026-12-31 Creates a review trigger
Replacement named candidate and test owner Makes retirement work planned rather than urgent

Enforce the catalog at admission and on every fallback

Resolve the requested model to one eligible route before the provider call. Then apply the same eligibility test to each fallback candidate. Do not convert a policy failure into a generic provider error: the caller needs to know whether it can retry, choose another approved model or ask for a policy change.

For example, an enterprise customer may allow a Claude model through its own Bedrock account for a document-review workload and allow a smaller OpenAI model only for classification. A timeout on the first route does not permit a cross-family fallback unless the second route also satisfies the workload’s catalog entry.

Alectura enforces model and provider choices through customer-matched policies. Its gateway contract documents model_not_allowed when no active connection serves a permitted model. It can retry eligible connections before the provider’s first byte, but it cannot make an unapproved route acceptable.

Test the ways an allowlist fails

Test four cases before a rollout:

  1. A provider retires an approved model while the replacement is still unapproved.
  2. A route is healthy but lacks a required feature, such as streaming or tool calls.
  3. The preferred connection fails and the next connection is in a disallowed region.
  4. An operator adds a broad model prefix that unintentionally admits a newer model generation.

Record the model ID, route, policy version and rejection reason for each decision. Keep prompt content out of that record when it is not needed. This gives support and security teams evidence without turning an allowlist into a new content-retention system.

Review catalog changes before they take effect

Separate provider discovery from permission changes. A scheduled job can identify a new model, a changed alias or a retirement notice, but it should create a review item rather than expand a customer catalog automatically. The reviewer compares capability, price, region, data terms and evaluation results with the workload record. If a change is urgent, use a short-lived exception with a named owner and a retest date. This preserves an evidence trail when a provider catalogue changes faster than an enterprise approval cycle.

Start with one high-value tenant and compare the effective catalog with the actual request logs. Then evaluate customer-specific policies with the quickstart using a non-production connection. The goal is a small, reviewable permission set that continues to hold during failure, migration and provider change.

Sources checked 14 November 2026. Provider capabilities and terms can change; check the linked documentation for your configuration.

Keep reading

Trace an internal AI incident without collecting every employee prompt ↗Build a secrets-redaction pipeline before AI requests leave your boundary ↗Validate LLM tool arguments before your application runs them ↗