Illustrative scenario. On a campaign Monday, the support agent that works beautifully the rest of the year receives in one hour what it normally gets in a day. The first messages go out fine. Twenty minutes in, the retries start, the queue grows, the model provider returns rate-limit errors, and the human inbox, meant to catch whatever the agent can't resolve, fills up all at once. Nobody broke anything: the system just never rehearsed that day.
The thesis in one line: an agent sized for the average day doesn't fail at the peak because it's bad; it fails because nobody decided beforehand what degrades first. You don't survive a spike with more horsepower; you survive it with a failure order agreed in advance.
Why do AI agents fail during demand spikes?
Because demand spikes don't stress what you tested (answer quality) but what you took for granted (everything around it). Four breaking points, almost always the same:
- Rate limits and quotas. The model provider caps how many requests and tokens per minute you get. If several agents share a key, one agent's spike eats everyone else's allowance. Real limits depend on your provider and plan: check them before, not during.
- Queues and retries. An unthrottled retry turns a one-off failure into an avalanche: every failed request re-enters and worsens the jam that made it fail.
- Cost. With more volume, longer conversations and more tool calls, spend grows faster than traffic. Without a cap you find out at month-end, when there's nothing left to decide.
- Swamped humans. The agent escalates hard cases to people; at the peak it escalates more, all at once. If the human inbox can't take it, the bottleneck doesn't disappear: it moves.
Notice what's missing from the list: the model. It's almost never the model. It's everything hanging off it, and that you can rehearse.
What should you rehearse before the spike?
Rehearsing means running, in good time, the small version of the disaster. You don't need a giant drill; you need an honest one. A minimum sequence:
- Measure your average day and set the expected peak using real data from past campaigns or closes, not gut feel. No history? Start from your worst known hours.
- Ramp the load in a test environment and watch where the first limit shows up: rate, queue, external tool or person. That's your real bottleneck, not the one you assumed.
- Test failure, not just load. Cut the external tool on purpose, simulate the provider's rate-limit error and watch what the agent does: if it makes things up, goes silent or retries forever, you know what to fix. Testing changes without breaking the automation works for this too.
- Set spend caps and per-system alerts, with a warning before the cap rather than after, and a named person who receives it during the peak.
- Rehearse the human handoff. How many cases per hour can the person on call absorb, and what happens to the rest? It's what almost nobody tests and what shows most; how you design human-in-the-loop decides whether that person is a brake or a net.
And measure while you rehearse: without detecting failures in your automations, a degraded spike can look like a quiet one.
What should you degrade on purpose so it doesn't fall over?
Degrading gracefully means deciding, before it's needed, what the agent stops doing first. It's the logic of the autonomy levels of an agent applied to time: at the peak the agent steps down a rung instead of falling off the building.
| What breaks | Possible degradation | What you protect |
|---|---|---|
| Provider quota exhausted | Switch to a lighter model for simple queries and keep the strong one for critical ones | Important cases keep getting answers |
| Growing queue | Instant acknowledgement with a realistic timeframe, processing in the background | Customer trust: they know their case exists |
| Cost over the cap | Limit expensive tools and conversation length; route the rest to a form | The budget, without switching the service off |
| Swamped humans | Triage by urgency and defer the non-urgent with a date, instead of escalating everything | The people, and the cases that can't wait |
None of this is glamorous, which is why it works: these are operations decisions, made calmly on a Tuesday in October, not at nine on Black Friday morning. If you'd rather have whoever runs it carry it, that's exactly what ensuring AI availability and monitoring AI in production cover.
Who decides what gets switched off during the spike?
An agreed degradation is only useful if someone has the authority to trigger it in the moment. Mid-spike there's no time to convene a committee, so the decision is made beforehand: you write one page with three things on it.
- The trigger. The concrete signal that activates each degradation: the queue passes a certain size, spend crosses a certain percentage of the cap, the provider returns rate-limit errors for several minutes in a row. Without a measurable trigger, every decision becomes a debate.
- The owner. A named person, plus a deputy, who can activate the degradation without asking anyone's permission. Authority delegated in advance is what turns a plan into a switch.
- The way back. How and when normal behaviour is restored, and who reviews whatever was deferred during the spike. A degradation nobody switches off becomes the new standard without anyone deciding it.
With that page, the spike stops being improvisation at nine in the morning and becomes a procedure. And what you learned that day gets written down for next time: what triggered each degradation, how long it lasted and what you'd do differently.
When don't you need to prepare for the spike?
When the peak is predictable and small, when the agent isn't on the critical path (if it goes down, people wait; no order is lost), or when being down for a while costs less than preparing. An internal lookup agent that's busier on Mondays needs no drill; one taking orders during a campaign does. Over-preparing is theatre too.
The line to take away: don't size for the average day or for the dream day; decide the order in which the bad day falls apart, and rehearse it.
Note: this article uses no third-party figures. Rate limits and pricing depend on the provider and plan, and they change; check your provider's documentation before sizing. The opening scenario is illustrative.