Docs

Gateway reference

Base URLs, API keys, identity headers, refusals and limits: the contract your servers call.

Your servers call the gateway in each provider's own API: you change the base URL, the API key and a few Alectura-* headers, and everything else is the provider's API, passed through unchanged. The management API is in openapi.yaml.

Base URLs

Base URLs
OpenAI SDKshttps://us.gateway.alecturalabs.com/openai/v1
Anthropic SDKshttps://us.gateway.alecturalabs.com/anthropic
Gemini SDKshttps://us.gateway.alecturalabs.com/gemini
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://us.gateway.alecturalabs.com/openai/v1",
    api_key=os.environ["ALECTURA_API_KEY"],
)
response = client.responses.create(
    model="gpt-5-mini",
    input="Say hello in French.",
    extra_headers={
        "Alectura-Organization": "initech",
        "Alectura-User": "user_42",
        "Alectura-Plan": "pro",
    },
)
print(response.output_text)
  • The two lines that change
  • Who the request is for

API keys

The gateway takes a sending key.

Key Format
Sending sk-alectura-{region}-{mode}-{secret}
Management sk-alectura-mgmt-{region}-{mode}-{secret}
  • {region} is us, eu or au. {mode} is live for an environment marked as production and test for any other.
  • {secret} is 40 random characters and then a 6-character checksum, all from [0-9A-Za-z]. The checksum is the CRC32 of everything before it, in base 62 and padded with zeros, so a scanner can tell a real key from a typo without calling us. Keys match sk-alectura-(?:mgmt-)?(?:us|eu|au)-(?:live|test)-[0-9A-Za-z]{46}, which is registered with GitHub secret scanning.
  • The key goes in whichever header the SDK already sends: Authorization: Bearer …, x-api-key: …, x-goog-api-key: … or api-key: …, on any path. Gemini's ?key= query parameter isn't read, so a key never sits in a URL; Google's SDKs send the header.
  • A key's type and region are read before lookup, so a management key gets permission_denied and another region's key wrong_region, naming the base URL to use, even if revoked.

Identity

Header Carries
Alectura-Organization Your own identifier for the organization the request is for
Alectura-User Your own identifier for the user
Alectura-Plan The plan the organization is on
Alectura-Conversation Optional. Requests that share it form a conversation; without it, those resending one opening and reply do.
  • Each value is 1 to 256 characters of printable ASCII, and case matters. A missing header is null, and policies can match null.
  • Only these headers set identity. Any other Alectura-* header besides those below, or one like X-Alectura-User or Alectra-User, is refused as invalid_request, naming the header meant, so a typo can't quietly make a request anonymous.
  • The provider fields that carry identity never set it: OpenAI's user, safety_identifier and prompt_cache_key, and Anthropic's metadata.user_id. A value a client sends in one of them goes upstream as its keyed hash, so equal values stay equal and prompt caching still works.
  • The gateway sends providers a safety identifier (safety_identifier in OpenAI's formats, user in its embeddings, metadata.user_id in Anthropic's) for a named user or organization: a keyed hash of the user, else of the organization, so a provider acting on abuse acts on one user, not your whole account. The hash key is per environment. Gemini's API has no such field.

The user's client

Header Carries
Alectura-User-IP Optional. Your user's IP address
Alectura-User-Agent Optional. Your user's User-Agent

key_sharing and shim_shape read only these. Bad values are ignored. Kept: a keyed hash of the /24 (or /48) and agent, and its first word.

Tags

Alectura-Tags: feature=search, env=prod labels a request: up to 10 key=value pairs, keys of up to 40 a-z, 0-9 or _ (lowercased), values as above without ,. A bad pair is refused. Records keep tags whatever the retention: put no content or personal data in them.

Endpoints

Method and path Format
POST /openai/v1/chat/completions OpenAI Chat Completions
POST /openai/v1/responses OpenAI Responses
POST /anthropic/v1/messages Anthropic Messages
POST /openai/v1/embeddings OpenAI embeddings
POST /anthropic/v1/messages/count_tokens Anthropic token counting
POST /gemini/{version}/models/{model}:generateContent Gemini
POST /gemini/{version}/models/{model}:streamGenerateContent?alt=sse Gemini, streamed
POST /bedrock/model/{model}/converse, or converse-stream Bedrock Converse
POST /bedrock/model/{model}/invoke, or invoke-with-response-stream Anthropic Messages, in Bedrock's envelope
GET /openai/v1/models OpenAI's model list
GET /anthropic/v1/models Anthropic's model list
GET /gemini/{version}/models Gemini's model list
GET any list's path, then /{model} One model of that list
GET /alectura/v1/models Every model's connections

LangChain sends embeddings as token ids, which no guardrail reads, unless check_embedding_ctx_length=False.

Cloud clients

Code that calls a cloud’s own client keeps it. The same request, through each:

import os

import boto3

os.environ["AWS_BEARER_TOKEN_BEDROCK"] = os.environ["ALECTURA_API_KEY"]
client = boto3.client(
    "bedrock-runtime",
    region_name="us-east-1",
    endpoint_url="https://us.gateway.alecturalabs.com/bedrock",
)
identity = {
    "Alectura-Organization": "initech",
    "Alectura-User": "user_42",
    "Alectura-Plan": "pro",
}


def identify(request, **_):
    request.headers.update(identity)


client.meta.events.register("before-send.bedrock-runtime.*", identify)
response = client.converse(
    modelId="us.anthropic.claude-haiku-4-5-20251001-v1:0",
    messages=[{"role": "user", "content": [{"text": "Say hello in French."}]}],
)
print(response["output"]["message"]["content"][0]["text"])
  • The lines that change
  • Who the request is for

AzureOpenAI takes azure_endpoint=https://{region}.gateway.alecturalabs.com/azure and api_key; Google's SDKs with vertexai=True, …/vertex_ai and api_key, or project, location and the key as credentials' token; AnthropicVertex, …/vertex_ai/v1 and access_token. A deployment is the model; project, location, api-version and labels are ignored.

AWS's SDKs and AnthropicBedrock take …/bedrock with our key as AWS_BEARER_TOKEN_BEDROCK, not SigV4. A model's ARN, or an application inference profile's (application_inference_profiles), reads as the model. Errors are Bedrock's; a block's x-amzn-ErrorType is AccessDeniedException:blocked_by_guardrail. InvokeModel serves only Claude; Converse serves every model.

The clouds' embeddings, token counting and model lists are refused.

Gemini

  • {version} is v1beta, the SDKs' default, or v1.
  • A stream is served only as server-sent events, with alt=sse, which Google's SDKs always send; without it, Google streams one JSON array, refused as invalid_request.
  • A thoughtSignature, and a function call or thought carrying one, pass byte for byte, since Google checks those: the args keep their pseudonyms, recorded encrypted. Signed answer text can still be edited.
  • The most output is generationConfig.maxOutputTokens.

Models

The model lists hold the models this request's policies allow that an active connection of the path's format serves, so they take the identity headers too. Ids are the providers' own, never renamed. For OpenAI's list, created is when the first connection serving the model was created. For Anthropic's, display_name is the id and the list is one page. The simulator refuses a Messages request without max_tokens, as the providers do.

GET /alectura/v1/models lists them in any format, by an id each listed endpoint takes, with connections: connection id, provider, region, the model id it lists, endpoints as records name them, and position, lowest first among those listing a request's endpoint. Errors are OpenAI's.

A policy names models by id. An id without a date matches every dated version of it, and Bedrock's and Vertex AI's ids match by the model they name, so claude-sonnet-4-5 matches claude-sonnet-4-5-20250929, eu.anthropic.claude-sonnet-4-5-20250929-v1:0 and claude-sonnet-4-5@20250929. A dated id matches only itself.

Routing

A request goes to a connection of its own format family, and no format is translated: OpenAI's formats to OpenAI, Azure OpenAI or Bedrock mantle; Anthropic's to Anthropic, or to Bedrock or Vertex AI running Claude; both to OpenRouter or Vercel AI Gateway; Gemini's to Gemini or Vertex AI; Converse to Bedrock. Only Anthropic counts tokens. Each connection's region says where its provider processes requests, from its endpoint (openapi.yaml). A body that sets models, fallbacks, route, provider, plugins or providerOptions is refused as invalid_request.

The policy's routing sets the candidate connections and their order: by model, region, the organization's own, then the policy's ranks, ties at random. A failure before the provider's first byte moves the request to the next, for up to three attempts, the last for a fallback (routing.models.add). After the first byte, there's no fallback.

Streaming

Each SDK streams as it does from the provider, with the client made as above:

stream = client.responses.create(
    model="gpt-5-mini",
    input="Say hello in French.",
    stream=True,
    extra_headers={
        "Alectura-Organization": "initech",
        "Alectura-User": "user_42",
        "Alectura-Plan": "pro",
    },
)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
  • A streamed response is the provider's own server-sent events, held until its first event; a failure before then is an HTTP error with its real status, and can fall back.
  • secrets and personal_data read the answer when their parts select output, tool calls included, so a shell command or patch is redacted or restored before the client runs it. To find a value split across deltas, they hold back the word being written and any before it that could begin a longer value (a number or a capital), so text reaches the client a few words late, even when they only record. Held text goes out as the format's own delta, carrying no usage, or in the chunk that finishes, never after it, so each official SDK reads one whole answer.
  • grounding reads the answer's prose and holds nothing back. It reads the sources beside the provider call, and the stream's end waits for that read.
  • While a guardrail holds back text, the gateway sends keep-alives: : keep-alive comment lines on the OpenAI formats and ping events on Anthropic's, as the providers do. Gemini has none, and Google's SDKs parse every line as JSON, so there it's a chunk with one empty text part: data: {"candidates":[{"content":{"role":"model","parts":[{"text":""}]},"index":0}]}.
  • So a 200 can still end in an error. A cut, or a provider failure after the first byte, ends the stream with the format's own terminal error:
# Chat Completions: a chunk carrying the error and an empty choice, then [DONE]
data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1790680024,"model":"gpt-5",
       "choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}],
       "error":{"message":"Stopped by the prompt_injection guardrail.","type":"alectura_policy",
                "param":null,"code":"cut_by_guardrail","request_id":"req_01M3PQJ64T…"}}

data: [DONE]

# Responses: response.failed, numbered one past the last event sent
event: response.failed
data: {"type":"response.failed","sequence_number":42,
       "response":{"id":"resp_…","object":"response","status":"failed",
                   "error":{"code":"cut_by_guardrail",
                            "message":"Stopped by the prompt_injection guardrail."}, …}}

# Anthropic Messages
event: error
data: {"type":"error","error":{"type":"permission_error","code":"cut_by_guardrail",
       "message":"Stopped by the prompt_injection guardrail."},"request_id":"req_01M3PQJ64T…"}

# Gemini: the error object Google sends when a stream fails, which its SDKs raise
data: {"error":{"code":403,"message":"Stopped by the prompt_injection guardrail.",
       "status":"PERMISSION_DENIED","details":[…]}}

A non-streamed response that a guardrail stops is a 403 refusal with the code cut_by_guardrail, since nothing has been sent yet; a call the provider hasn't answered is dropped.

Refusals

A refusal comes back in the format's own error shape, so your error handling already works, and the provider never sees the request.

{
  "error": {
    "message": "Requests for this user are suspended.",
    "type": "alectura_policy",
    "param": null,
    "code": "user_suspended",
    "request_id": "req_01M3PQJ64T…"
  }
}
{
  "type": "error",
  "error": {
    "type": "permission_error",
    "code": "user_suspended",
    "message": "Requests for this user are suspended."
  },
  "request_id": "req_01M3PQJ64T…"
}
{
  "error": {
    "code": 403,
    "message": "Requests for this user are suspended.",
    "status": "PERMISSION_DENIED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        "reason": "user_suspended",
        "domain": "alecturalabs.com",
        "metadata": {}
      },
      {
        "@type": "type.googleapis.com/google.rpc.RequestInfo",
        "requestId": "req_01M3PQJ64T…"
      }
    ]
  }
}

On the OpenAI formats type is always alectura_policy, a bad key's too: OpenAI's SDKs raise their authentication error from the 401 alone, and code says which refusal it is. On Anthropic's, type is Anthropic's own for the status, and code is ours. On Gemini's, code is the status and status Google's own name for it, as Google's errors have them; our code is the reason of an ErrorInfo in the domain alecturalabs.com, and the request id is a RequestInfo, the places Google's own APIs put them. Each code has one status, and only a code that waiting may fix gets a status SDKs retry on their own (429 and 5xx). A fault of the gateway's own is a 500, as each provider answers its own.

Status Anthropic type Gemini status Code When
400 invalid_request_error INVALID_ARGUMENT invalid_request The body can't be parsed, so the guardrails can't run; or it picks models or providers no policy checks; or an Alectura-* header is unknown or invalid.
400 invalid_request_error INVALID_ARGUMENT conflicting_api_keys The request carries two different keys.
401 authentication_error UNAUTHENTICATED invalid_api_key No key, or an unknown, expired or revoked one.
402 billing_error FAILED_PRECONDITION budget_exhausted A limit over a day, week or month is spent. The error names the limit and when it resets.
403 permission_error PERMISSION_DENIED permission_denied The key can't send, such as a management key.
403 permission_error PERMISSION_DENIED wrong_region The key belongs to another region.
403 permission_error PERMISSION_DENIED user_suspended, organization_suspended A suspension.
403 permission_error PERMISSION_DENIED model_not_allowed, region_not_allowed, no_allowed_connection No active connection serves the model, or the routing settings allow none; or a spend limit applies and the model has no list price, so its cost can't be counted.
403 permission_error PERMISSION_DENIED blocked_by_guardrail, attachment_uninspectable, cut_by_guardrail A guardrail blocks the request, or the policy blocks content nothing can inspect, such as an image or unknown part, or a guardrail stopped a non-streamed response.
404 not_found_error NOT_FOUND unsupported_endpoint A provider path the gateway doesn't serve.
413 request_too_large INVALID_ARGUMENT request_too_large The body exceeds the provider's cap: 32 MB for Anthropic, 72 MiB for OpenAI, 100 MB for Gemini.
429 rate_limit_error RESOURCE_EXHAUSTED rate_limited A per-minute limit, or a trial's on the simulator, with Retry-After in seconds.
500 api_error INTERNAL internal_error A fault in the gateway itself. The request is recorded with the policies that matched.
502 api_error UNAVAILABLE provider_unreachable Every candidate was unreachable or disconnected.
503 api_error UNAVAILABLE limiter_unavailable Spend so far isn't known, so a spend limit can't be checked.
503 api_error UNAVAILABLE gateway_unavailable The gateway can't check the request yet: suspensions aren't loaded, or a store it reads, such as the environment's keys, failed.
504 timeout_error DEADLINE_EXCEEDED provider_timeout No candidate answered in time.

A spent budget is FAILED_PRECONDITION rather than Google's RESOURCE_EXHAUSTED, which Google uses for its own rate limits and its SDKs retry.

A refusal over a limit adds a limit object beside code, or on Gemini's format the same fields as limit_kind, limit_per and so on in the ErrorInfo's metadata, which holds only strings:

"limit": {
  "kind": "spend",
  "per": "user",
  "period": "day",
  "amount": "2.00",
  "used": "1.9962",
  "requested": "0.0079",
  "resets_at": "2026-09-30T00:00:00.000Z"
}

used is what the period has used so far, and requested what the refused request would have used at most: its estimated cost, its input and maximum output tokens, or one request. A request is refused when the two together would pass amount, so it can be refused before used reaches it. A limit on tagged requests adds their tags (limit_tags, as JSON, on Gemini's format).

Provider errors

An error the provider returns passes through unchanged: its status, its body and its headers. Three exceptions:

  • Anything in it that exposes a connection is masked with ***: an endpoint URL, an AWS account or role, the provider's organization, project or account id, or a credential the request carried, such as the session token a Bedrock signature error quotes.
  • Bedrock's errors take Anthropic's shape for a client of Anthropic's API, and Anthropic's take Bedrock's for a client of InvokeModel, keeping the status.
  • A success whose body, or whose stream's first event, is only an error, as OpenRouter sends when a model fails after its keep-alives, is a failure: it can fall back, and the client gets the error as a 502.

When every attempt fails, the client gets its model's last failure: the provider's error, or ours if it never answered.

Headers

Header Direction What happens
Authorization, x-api-key, x-goog-api-key, api-key in The Alectura key; never sent upstream
Alectura-* in As above; never sent upstream
anthropic-version, anthropic-beta in Passed to the provider
OpenAI-Organization, OpenAI-Project in Dropped: the connection names the provider account and project
Other request headers in Passed to the provider, except hop-by-hop headers and those a proxy adds: Forwarded, X-Forwarded-*, Via, X-Real-IP and X-Amzn-Trace-Id
Alectura-Request-Id out Alectura's request id, on every response and as an error body's request_id; an SDK's _request_id is the provider's, or Alectura's on a refusal
Alectura-Connection, Alectura-Region out The connection's id and region, when a provider was reached
Retry-After out On rate_limited
The provider's response headers out Passed through, rate-limit headers included, except those naming the provider account

Limits

  • A limit over a day, week or month resets when the next UTC period starts; weeks start on Monday.
  • A request reserves its estimated tokens and cost when it's admitted, at the connection's list prices, and settles them from the provider's usage when it ends; when the provider reports none, at its estimated input and the text the client got, flagged usage.estimated.
  • An embedding reserves no output. A token count is a request with no tokens or cost.

Compatibility

  • A new field or event in a provider's API passes through without a gateway release.
  • Once its key is known, a request is recorded whatever happens to it, refusals included; see GET /v1/requests.