A per-user LLM budget can look correct in a dashboard and still fail under parallel requests. The common mistake is to check the remaining balance before inference, then record the charge after the response. Every concurrent request sees the same unspent allowance and is admitted independently.
To reduce this gap, reserve an estimate before admitting each request and reconcile it with the eventual charge. The reservation decision must be atomic within the boundary that enforces the limit. This article explains the pattern and its failure cases; its numbers are hypothetical and are not model pricing.
Show the race with a small example
Suppose a user has 10 example cost units remaining. Four requests arrive together, each estimated to cost 4 units. A check-then-charge design admits all four because each sees the original balance. Their eventual combined charge can be 16 units.
A reservation design admits the first two requests and holds 8 units. The third request sees only 2 units available and is refused or deferred. The fourth gets the same result. This holds only if the read and reservation update are one operation, rather than another pair of independent reads and writes.
Use fixed-point amounts or integer cost units for accounting. Document the unit and rounding rules. Floating-point arithmetic and inconsistent conversions can create disagreements between the admission counter, settlement record and the balance displayed to the user.
Keep rate, token and spend controls distinct
Request rate limits constrain how frequently a person can start work. Token limits constrain quantities of model input or output. Spend limits constrain a cost estimate that depends on the model and its applicable pricing. A small number of expensive requests can pass a rate limit while exhausting a budget.
OWASP identifies uncontrolled inference as unbounded consumption, including resource exhaustion and financial impact. Its mitigations include quotas, rate limits and resource management. For your product, turn those categories into explicit admission rules tied to authenticated application identities.
Apply an organization-level allowance when multiple users share a commercial budget. A per-user limit alone is easy to multiply by creating accounts. Also constrain request size, output allowances and concurrent work where those dimensions matter to cost or service availability.
Estimate before admission, then settle once
A reservation should reflect the request being admitted, including the input and permitted output. The estimate is not a guarantee of the provider's eventual charge. Features with additional billing dimensions need explicit treatment rather than being silently counted as ordinary text tokens.
The following is architectural pseudocode, not an Alectura API or a complete distributed accounting implementation:
admit(request_id, user, estimate):
atomically:
return existing decision if request_id was already admitted
refuse if spent + reserved + estimate exceeds allowance
persist reservation for request_id
send request to provider
settle(request_id, actual_cost):
atomically:
do nothing if this reservation was already settled
remove its reserved estimate
add actual_cost to spent
persist the settlementThe duplicate checks are part of the accounting contract. A retried callback should not charge twice, and an application retry should not accidentally reuse an identifier for different work. Define whether a provider retry is part of one admission or a separate billable attempt.
Decide what happens when usage is missing
A client can disconnect before the provider finishes. A worker can restart after admission. A response may arrive without the usage detail your settlement code expects. These outcomes require an explicit reconciliation policy.
Do not release a reservation merely because the client stopped listening. The provider may already have processed billable work. Keep enough state to recover the outstanding request and apply a conservative documented estimate when exact usage cannot be obtained. Record that it is estimated so operations can investigate discrepancies.
Test failures between each durable state transition. A process that dies after reserving but before dispatch should eventually release or reconcile the hold. A process that dies after dispatch must not assume that no cost occurred. The recovery design needs to distinguish these cases as far as the available evidence permits.
State the boundary of the limit
An in-memory counter enforces only what shares that memory. Multiple gateway instances can admit against separate counters unless the design coordinates them. Deployments can temporarily create the same condition even when normal operation uses one instance.
Alectura reserves estimated usage at admission, but its current single-gateway design has an important rollout limitation: during a deployment, two tasks can temporarily count separately. A limit can admit up to twice its configured amount in that interval. Treat that as a stated operational boundary, not a hard financial ceiling. The limits overview should be assessed with the deployment behavior of the system you run.
Before enabling paid access, run a concurrency test, retry the same settlement, interrupt a stream and restart the enforcing process. Compare reservations, settled amounts and request records. A useful spend-control design is one whose exceptions you can explain and recover, as well as whose normal path rejects the next request at the expected point.
Use the Alectura quickstart to evaluate the request boundary in a non-production environment. Test the parallel-request and interrupted-response cases against your own allowance before enabling paid access.