Rate limiting & cost controls
DioscHub enforces two independent limits. Token budgets cap how much model consumption an individual can drive — the cost control. Rate limits cap how many requests and connection events traffic can make — the abuse control. They are configured separately and do different jobs; this page covers both.
Token budgets
Section titled “Token budgets”A token budget caps the number of tokens an individual can consume within a time window. Budgets are measured in tokens, never in currency — DioscHub does not price model usage in money.
Who a budget applies to. The individual is the Session for an anonymous user, and the authenticated user once they sign in. Two people sharing one Assistant each get their own budget; they do not share one pool.
The window is rolling. A window opens on first use after a quiet period, stays open for its duration, and a fresh window begins on the next use after it closes. It is not a fixed calendar reset.
What happens at the cap. The next turn is blocked and the user is told, rather than being silently throttled. There are two cases: the budget is already spent, or the remaining budget is too small to fit the prompt — in which case the user is prompted to start a new chat.
Where budgets are set
Section titled “Where budgets are set”A budget resolves from the most specific source available, falling back outward:
- Role — a Role’s budget, if set, overrides everything below.
- Assistant — the Assistant’s own budget.
- Platform default — a per-audience default when neither above is set.
The Assistant’s anonymousMode selects which platform default applies: an anonymous-facing Assistant
falls back to the customer default, an internal one to the employee default. You set these on
the Assistant and Role configuration, and tune the platform defaults in system settings.
On each entity the fields are tokenBudget, tokenBudgetWindowMs, and tokenBudgetUnlimited. A zero
or unset budget means “inherit the next level” — it never means unlimited. To grant unlimited, set
tokenBudgetUnlimited.
What the assistant knows about the budget
Section titled “What the assistant knows about the budget”The assistant can answer “what is my quota?”, “when does my limit reset?” and “how much have I used?” from the real numbers. It has two ways to know them. Both work on every tier and need no licence feature and no setting.
The get_my_usage tool. A built-in, read-only tool in the pinned usage toolset. It takes no
arguments, needs no approval, and costs no tokens until someone asks. It reads the same figures as the
usage bar in the chat widget, so the two agree at the same moment. It returns:
| Field | Meaning |
|---|---|
used, limit, remaining, percentUsed | Tokens in the current window. |
resetsAt, windowMs, windowType | When the window closes (ISO time), its length, and rolling or sliding. |
unlimited | true when no cap applies. |
limitSource, windowSource | Where the limit and the window length come from: role, assistant or platform_default. The two are resolved separately, so they can differ. |
scope | user for a signed-in user, session for an anonymous visitor. For a visitor the assistant says “this conversation”, not “your account”. |
context | How full the conversation’s context window was on the last model call: usedTokens, windowTokens, percentUsed. usedTokens counts the whole model input (system prompt, tool schemas and history). |
note | One plain sentence for the edge cases below. |
- Unlimited.
unlimitedistrueand the note says no limit applies. - No window open.
usedis0,resetsAtisnull, and the note says the window starts with the next message. - Sliding window. There is no fixed reset. The note says usage frees up as older spend ages out.
- Read failed. The tool says usage is unavailable. It never quotes a made-up number, and it leaves out
context.
The tool is offered when a budget resolves for the user’s session, an unlimited budget included. It is not offered when none resolves. This is checked on every turn. A short line in the assistant’s instructions tells it to call the tool for questions about its own quota, usage, limit or reset time.
The status block. Below 80% of the quota the assistant is told nothing, so it does not bring the subject up on its own. Past a level, the hub adds a short status block to the request:
- 80% and 95% of the limit (fixed hub constants, not settings). The turn that first crosses a level carries the percent used, the tokens remaining and the reset time. The assistant is told to open its reply with a one-sentence heads-up, once. Later turns at the same level carry only an instruction to keep answers concise, with no numbers. Crossing 95% after 80% warns again. When usage falls below 80% (a new window), the levels reset.
- Context trigger, with or without a budget. It fires when the conversation has been compacted, or when the prompt comes within about 400 tokens of leaving no room for a reply. The assistant is told to suggest starting a new chat, once per session.
The heads-up instruction is added to a copy of the user’s latest message, labelled [Usage status from the hub], for every model provider. It is not saved in the chat history. The rest of the block stays out
of the cached system prompt. If a turn fails or is cancelled before the assistant replies, the warning is
not counted as given and comes again on the next turn.
The tool output and the block carry numbers, levels, times and source labels only. They never carry a user id, an email, an auth header or any credential, so the assistant stays credential blind. The widget’s usage bar is unchanged and remains the reliable warning; the assistant’s heads-up is a courtesy on top of it.
Request & connection rate limits
Section titled “Request & connection rate limits”Separately from token budgets, DioscHub rate-limits raw traffic:
- HTTP requests — a per-second burst limit and a sustained hourly limit, with tighter sub-limits
on upload and download paths, and a separate per-minute limit on a user’s file Re-index
(
REINDEX_LIMIT_PER_MINUTE, default 5; environment only). Over it, Re-index answers429with aRetry-Afterheader. See Chat files & indexing. - WebSocket events — tiered limits on model calls, resumes, session operations, connections, and overall event volume.
These count requests and events, not tokens. Requests are attributed to the authenticated user when
known, otherwise to the client IP. Defaults are configurable in system settings and via environment
variables (THROTTLE_BURST_LIMIT, THROTTLE_SUSTAINED_LIMIT, and the upload/download limits), and can
be overridden per deployment through the admin rate-limit settings.
Requests to the admin API are exempt from the per-second burst cap so the portal stays responsive; the hourly sustained cap still applies to them. Public, embed, and anonymous traffic is fully burst-limited.
Anonymous session limits
Section titled “Anonymous session limits”Anonymous visitors also have a cap on how many new chats they can start: by default 5 per hour per browser and 60 per hour per network (client IP), for each Assistant. These caps stop a visitor from resetting their per-session token budget by opening chat after chat. Over a cap, the chat is not started and the widget tells the visitor how long to wait. See Anonymous session limits.
Across a cluster
Section titled “Across a cluster”In a single instance, request and WebSocket limits are counted in process. In cluster mode, those counters move to the shared Redis backplane, so a user’s request and event limits hold across every instance.
Token budgets are counted against a shared usage ledger in Postgres, so an individual’s spend is tracked cluster-wide — though each instance keeps a short-lived read cache, so a burst spread across instances within the same moment may be counted with slight lag. There is, as above, no aggregate cross-user token ceiling in either topology.