Build your approved LLM inventory around an application, a purpose and a deployment. For each approval, record which model or version the application can use, which provider route it can use and under what conditions. Compare those records with both the routes you configured and the routes that served requests. A catalog of model names alone cannot show whether applications still use the routes you reviewed.
Define the approval record
Start with a stable approval ID and the application and business owners. Record the provider, model identifier, deployment or project, allowed processing destination, permitted data classes, required controls and approved purpose. Include the decision evidence, review date and expiry if applicable. Link to the assessment and evaluation report so you can trace the decision without copying those documents into every approval row.
NIST AI RMF Govern 1.6 covers AI system inventories and describes artifacts and contacts that help with maintenance and incident response. A model approval register can serve as a dependency view of that larger inventory. Keep the application ID stable so the platform team can find every application affected by a route change and contact the owner who can validate a replacement. NIST AI RMF Playbook
Record different decisions for the same model
Suppose two hypothetical applications use a fictional model called Atlas Text v2. The platform team reviews each provider route against the permitted data and processing destinations for that application.
| Application | Route and scope | Approval disposition |
|---|---|---|
| IT help assistant | Regional deployment, internal help articles | Approved after evaluation against the help corpus. |
| Procurement review assistant | Same regional deployment, contract extracts | Pending review of contract handling and sensitive-data controls. |
| IT help assistant fallback | Alternate provider deployment, internal help articles | Pending evidence of the processing destination. |
When IT adds the fallback, require a decision on that row before it receives production traffic. A regional route review is evidence for that route only. If your business rules allow a model alias, record the acceptable versions or track the resolved model wherever your provider exposes it. When response metadata and deployment configuration cannot establish the effective version, record that limitation and decide whether the use case can proceed. Provider destination review
Compare approvals with configuration and traffic
Compare your application-to-route configuration with current approval rows, starting with routes that can receive sensitive data. Include all eligible fallback models. Compare actual provider, model and route records with approvals for the same time window. Configuration can expose routes that saw no traffic during the window. Observations can expose a stale deployment or a call path missing from the configuration export. Keep a list of known direct integrations outside the gateway and use provider records or application telemetry to cover them.
Review an unrecognized served route for unexpected data exposure. For a configured route with no matching approval, remove it from the application's eligible configuration or complete the missing review. When an approval applies only to one data class, model and route metadata confirms part of the rule. You still need evidence from the application or a data control that the request used that class.
Track provider lifecycle in a separate field. Amazon Bedrock's model lifecycle API exposes active or legacy status and can include an end-of-life timestamp. A retirement triggers a review of affected applications. A provider returning an active status shows it considers the model available, while the internal record still determines whether the application may use it. Amazon Bedrock model lifecycle API
If you route through Alectura, compare the model allowlist and provider-region policy for the relevant customer with the approved routes. Try one allowed request and one request for a disallowed model through the gateway before rollout. Check application telemetry to cover requests on a different path. Alectura gateway documentation
Source documents (2)
- NIST AI RMF PlaybookChecked
- Amazon Bedrock model lifecycle APIChecked
Trace a request through Alectura.
Bring a sample prompt and a customer policy. See what reaches the model and which decisions appear in the request record.
