Choose fail-closed behavior for inference policy failures

Decide when an AI SaaS application should stop a request because policy evidence is unavailable, and design a truthful recovery path.

Fail closed when the system cannot establish a policy condition that protects a request. For an AI SaaS application, that commonly includes an unavailable identity-to-tenant lookup, missing allowlist configuration, an unverified destination, or an unavailable limit reservation. Apply the same rule to a required guardrail when its result determines whether the request may proceed.

This choice has a cost: a dependent store outage can stop a useful feature. The alternative can send a customer’s data to an unapproved model, admit unbounded spend, or execute a request under the wrong tenant policy. Make the choice by policy class, then give the client a recovery path that does not disguise a protection failure as an upstream timeout.

Choose an action for each failed control

One service can support both fail-closed and fail-open decisions. The decision depends on what becomes possible if the check is skipped.

Failed dependency or check Safer default Reason Recovery path
Tenant identity or policy lookup Fail closed The system cannot select the customer’s constraints Retry after a short delay; show a policy-service error
Model and destination allowlist Fail closed An unreviewed provider path could receive data Use a pre-approved connection only after policy lookup recovers
Spend or concurrency reservation Fail closed for a hard limit Parallel requests can exceed the agreed limit Retry after Retry-After or use an approved lower-cost path
Optional usage analytics Fail open only if it is not an enforcement record The request policy is still proven Mark the observability gap and alert operators
Cosmetic response enrichment Fail open It does not alter authorization or data handling Return the base response

Do not create a “temporary bypass” that an ordinary request can trigger. If an operator must restore a service under exceptional conditions, use a separate, audited change with a defined expiry. Otherwise the bypass becomes the effective production policy during every incident.

Separate a refusal from an availability error

A request can be refused because it violates a known policy, or it can be unavailable because the policy cannot be evaluated. Clients need different behavior. A known refusal should not retry automatically. An availability failure may be retried within a deadline only when a repeated request cannot create an unsafe or duplicate side effect.

Record a stable reason code, request identifier, policy version when known and whether any provider attempt started. Do not put a provider credential, prompt contents or internal topology in the user-visible error. For streamed calls, make the admission decision before the first response byte where possible. Once text or a tool proposal reaches the application, the retry and user-interface rules become more complex.

OWASP’s prompt-injection prevention guidance recommends treating untrusted content and tool use as security boundaries. This supports a fail-closed decision when an authorization control cannot run. A system prompt that says “do not perform the action” is not a substitute for an independent check.

Use a worked outage scenario

Assume a paid SaaS feature sends a support ticket to a model after determining the customer’s allowed region and maximum spend. The policy store times out after the request has authenticated. A fail-open design uses a default global connection. That preserves availability but may violate the customer’s agreed restriction. A fail-closed design returns a retryable policy-service error before provider dispatch and records no provider attempt.

The team can reduce user impact without weakening the rule: cache a signed, short-lived effective policy; isolate the policy store; provide a status page; and use a client retry budget. Each measure must have an expiry and invalidation model. An indefinite cache can continue to allow a revoked customer or retired model.

Test both sides of the boundary

In a non-production environment, make the policy dependency unavailable. Verify that a hard-limit or route policy request fails before any outbound provider call. Then remove an optional telemetry sink and verify that the request proceeds only if the remaining enforcement evidence is intact. Test concurrent retries so a client loop cannot turn a temporary failure into a burst.

The OWASP system-prompt leakage guidance also states that critical authorization bounds should be enforced independently from the model. Treat that as a design constraint: service availability work must not move the authorization decision into a prompt.

Alectura uses stable refusal codes and records attempts; its routing and limits reference describes policy bounds and reservations. Confirm the specific refusal behavior with your integration and model SDK. Fail-closed design reduces a defined policy risk. It does not make an AI feature continuously available or eliminate the need for ordinary incident response.

For a related guardrail failure mode after a stream has started, see how to handle policy refusals and interrupted streams.

Source documents (2)

Trace a request through Alectura.

Bring a sample prompt and a customer policy. See what reaches the model and which decisions appear in the request record.

Try the quickstart

Keep reading

How to Measure LLM Prompt Cache Savings ↗Set Up Customer-Owned LLM API Keys in AI SaaS ↗Which AI outputs need a label under EU AI Act Article 50? ↗