Set Up Customer-Owned LLM API Keys in AI SaaS

Design a tenant-scoped provider connection, with checks for credential scope, billing attribution, revocation and fallback.

If your SaaS customer supplies an LLM API key, bind the key to that customer's connection on the server. Resolve the connection from the authenticated customer, feature and environment on every provider attempt. Keep the customer-owned route distinct from a route that bills your SaaS account. Otherwise, an outage or an expired key can change which provider account pays for a request with no visible change in the application.

Give each connection an owner and a scope

Record the customer ID, environment, provider, project reference, approved destination, permitted models and secret reference. Store the key itself in your secrets system. Give the runtime access to the credential through the connection it resolves, and limit who can create, test or disable the connection. Record administrative changes against the connection ID so you can audit them without exposing the secret.

Ask your customer for a dedicated service credential scoped to the project and operations the integration uses. OpenAI service accounts belong to a project, and OpenAI documents controls for restricting key access after creation. Verify that your integration can call its required endpoint with the restricted key. Confirm which project owns the resulting usage. A successful inference call alone cannot tell you whether the customer supplied a production credential or a sandbox credential. OpenAI project settings

Your application still needs tenant isolation. AWS describes using the current tenant context to constrain resource access even after a user has authenticated. Apply that restriction when an administrator selects a connection and when a background worker resolves it. A worker that receives a queued connection ID should recheck the association against the job's customer ID before it loads the secret. AWS tenant isolation guidance

Decide which attempts may proceed

Suppose a hypothetical contract-review SaaS serves two customers. North uses its own production project for document summaries. South uses the SaaS provider project. Both features display the same model name.

Event Expected result
North requests an approved model with its connection active Use North's connection and record its provider project.
North's credential is revoked while a job is waiting Recheck the connection at dispatch. Stop the job and notify the connection owner.
North's provider throttles the request Retry within policy or use another North connection approved for the data, model and destination.
South calls the same model Use the SaaS-owned route and attribute the attempt to South.

Reject a request when its required customer-owned route is unavailable. If you sell a managed fallback, obtain approval for the separate route and billing arrangement before you enable it. Include the fallback's processing destination and data rules in that decision. An identical model ID can represent a different provider deployment or account.

Verify the invoice boundary

Keep the connection ID, attempt result, provider request ID when available, served model and usage counters in your operational records. Use those fields to distinguish your SaaS invoice exposure from customer-owned provider spend. In a test where the first attempt fails and the alternate succeeds, check both records against the corresponding provider projects. Preserve an unknown usage state when the provider returns no counters. Compare customer-attributed usage with available provider reports before using estimates for any financial total. Customer-level LLM cost reconciliation

Test connection selection with two customer fixtures and reverse their execution order to catch a credential cache keyed only by model or provider. Run a queued job before and after disabling its connection. Verify that the later job makes no provider attempt on the disabled route. Use the provider credential rotation guide to test key replacement separately.

Alectura uses provider accounts and credentials that you configure and supports testing a connection before use. Customer-level model and provider-region policy governs eligible routes. In a pilot, use distinct non-production connections and verify the eligible model and destination for each request. Check credential ownership and provider billing in your integration before enabling the route for production traffic. Alectura quickstart

Source documents (2)

Trace a request through Alectura.

Bring a sample prompt and a customer policy. See what reaches the model and which decisions appear in the request record.

Try the quickstart

Keep reading

How to Measure LLM Prompt Cache Savings ↗Which AI outputs need a label under EU AI Act Article 50? ↗When to Buffer Streaming AI Output for Guardrails ↗