Rate limiting & cost controls

DioscHub enforces two independent limits. Token budgets cap how much model consumption an individual can drive — the cost control. Rate limits cap how many requests and connection events traffic can make — the abuse control. They are configured separately and do different jobs; this page covers both.

Token budgets

A token budget caps the number of tokens an individual can consume within a time window. Budgets are measured in tokens, never in currency — DioscHub does not price model usage in money.

Who a budget applies to. The individual is the Session for an anonymous user, and the authenticated user once they sign in. Two people sharing one Assistant each get their own budget; they do not share one pool.

The window is rolling. A window opens on first use after a quiet period, stays open for its duration, and a fresh window begins on the next use after it closes. It is not a fixed calendar reset.

What happens at the cap. The next turn is blocked and the user is told, rather than being silently throttled. There are two cases: the budget is already spent, or the remaining budget is too small to fit the prompt — in which case the user is prompted to start a new chat.

Where budgets are set

A budget resolves from the most specific source available, falling back outward:

1. Role — a Role’s budget, if set, overrides everything below.
2. Assistant — the Assistant’s own budget.
3. Platform default — a per-audience default when neither above is set.

The Assistant’s anonymousMode selects which platform default applies: an anonymous-facing Assistant falls back to the customer default, an internal one to the employee default. You set these on the Assistant and Role configuration, and tune the platform defaults in system settings.

On each entity the fields are tokenBudget, tokenBudgetWindowMs, and tokenBudgetUnlimited. A zero or unset budget means “inherit the next level” — it never means unlimited. To grant unlimited, set tokenBudgetUnlimited.

Note

A token budget limits one individual within one window. DioscHub does not enforce an account-wide or fleet-wide total-token ceiling across all users. If you need an aggregate cap on total consumption, budget it at your LLM provider.

Request & connection rate limits

Separately from token budgets, DioscHub rate-limits raw traffic:

• HTTP requests — a per-second burst limit and a sustained hourly limit, with tighter sub-limits on upload and download paths.
• WebSocket events — tiered limits on model calls, resumes, session operations, connections, and overall event volume.

These count requests and events, not tokens. Requests are attributed to the authenticated user when known, otherwise to the client IP. Defaults are configurable in system settings and via environment variables (THROTTLE_BURST_LIMIT, THROTTLE_SUSTAINED_LIMIT, and the upload/download limits), and can be overridden per deployment through the admin rate-limit settings.

Requests to the admin API are exempt from the per-second burst cap so the portal stays responsive; the hourly sustained cap still applies to them. Public, embed, and anonymous traffic is fully burst-limited.

Across a cluster

In a single instance, request and WebSocket limits are counted in process. In cluster mode, those counters move to the shared Redis backplane, so a user’s request and event limits hold across every instance.

Token budgets are counted against a shared usage ledger in Postgres, so an individual’s spend is tracked cluster-wide — though each instance keeps a short-lived read cache, so a burst spread across instances within the same moment may be counted with slight lag. There is, as above, no aggregate cross-user token ceiling in either topology.