Skip to content
Implementa.

Automating with AI · Guide 19 of 19

Who responds when an automation goes down: on-call, severities and runbooks for a small company

The automation has been running for eight months and nobody has mentioned it since. That is exactly what you paid for — and also the reason nobody has ever answered the question that matters: if this goes down, who picks it up? The question ends up answering itself, badly, at 11:40 on a Friday night, with a message to the team group chat and fourteen orders unprocessed that nobody knows exist yet. This guide is about answering it beforehand, cold, and doing it at the scale of a small company: no three-tier rotations, no pagers, and no copying the playbook of a big tech company with a hundred engineers you do not have.

The question nobody answers until it is 11:40 on a Friday night

The automation has been running for eight months and nobody has mentioned it since. That is exactly what you paid for. The problem is that the silence gets mistaken for an answer to a question nobody ever asked: if this goes down, who picks it up? And the question answers itself, badly, the first night the order flow stops creating delivery notes: somebody messages the team group chat, somebody else replies at seven in the morning, and along the way fourteen orders have gone unprocessed that nobody knows exist yet.

On-call and support for automations is the day-two work almost nobody buys, because it does not show up in the demo. It is not a 24/7 desk with a switchboard. It is four decisions made cold —what counts as serious, who gets called, what that person does when they arrive, and what happens in the meantime— written somewhere findable by someone who is not you. Without them, your contingency plan is your memory, and your memory takes August off.

Real severities: what justifies waking someone up and what waits until morning

The first decision is not who you call: it is when anyone gets called at all. Without that line written down, both bad things happen at once. Everything looks urgent —and the team learns to ignore the alerts because they fire for anything— or nothing does, and the failure that was costing money gets discovered on Monday.

The operations industry has spent decades solving this with severity levels, and the established practice is sound: PagerDuty recommends mapping each level directly to the escalation and notification policy, and warns that six or seven levels nobody can tell apart produce classification paralysis: three to five is the sweet spot. In a company with no on-call team, three is plenty.

LevelWhat it means at your placeWhat happens
SeriousMoney or a customer commitment is being lost right now: orders are not going out, invoices are not getting charged, nobody is getting answered.The on-call person is paged on the spot, whatever the hour. The manual fallback kicks in.
AnnoyingThe process half works, or only works for part of it: an integration down with a workaround, a batch that did not process but can be reprocessed.Logged and picked up first thing the next business day. Nobody gets called at night.
Ugly but it can waitSomething is wrong and there is no immediate consequence: a report that did not generate, a field arriving empty somewhere nobody looks at until month-end close.Goes into the week's maintenance queue.

The criterion that makes this table work is not technical, it is commercial: severity is defined by the damage done for every hour that passes, not by how broken the system is. A completely dead flow that only moves an internal report is "ugly but it can wait". A flow running at 90% that is sending quotes with the wrong price is serious. Write the table with that logic and with your processes by name —"web orders", "invoicing", "customer notifications"— not with abstract categories. A severity level that has to be interpreted at 11:40 at night is worth nothing.

The on-call that fits a small company (do not copy the big tech model)

This is where most articles turn useless: they describe weekly rotations, three-tier escalation and pagers. You have two people who know how to touch the flows and one of them is you. On-call that is realistic at that scale has four pieces and none of them costs money.

  1. One name per week, written down. Not "the team", not "whoever sees it". One specific person per week, on a shared calendar, with the weeks already assigned through the end of the quarter. The value is not in the rotation: it is that at 11:40 at night nobody has to decide who to bother.
  2. A channel that actually interrupts. Email does not wake anyone up and a muted Slack channel does not either. A serious-severity alert has to arrive through something that makes noise: a phone call, a text, a notification set as a do-not-disturb exception. Everything else can go through the usual channel.
  3. One backup, and only one. If the on-call person does not answer within a window you decide —fifteen minutes is reasonable—, the backup gets paged. No four-tier chain: in a twenty-person company, the third tier is always the same person.
  4. An explicit compensation rule. If somebody handles an incident on a Saturday, it gets paid or it gets taken back as time off. On-call built on goodwill lasts exactly until the second time somebody loses a dinner.

And one honest decision before you build anything: do you actually need overnight on-call? If your critical processes only have consequences during business hours —invoicing, B2B orders, internal notifications—, the right answer is to have no night on-call and say so out loud, instead of running an informal rotation nobody ever agreed to. What you do always need is the failure logged and visible first thing in the morning. In n8n that is built with a single error flow wired to all the others: the official documentation describes how one Error Workflow with an Error Trigger node gets reused across every production flow, instead of building the alert one by one. Make and Zapier have their own equivalents.

The one-page runbook: the only thing that lets someone else respond

On-call without a runbook is a phone ringing to tell somebody they cannot do anything. A runbook is the one-page card —one, not twelve— that turns "the order flow is down" into a sequence that can be executed by whoever did not build it. PagerDuty says it without decoration: high-severity incidents should point to a runbook so the responder does not improvise.

One per critical flow. Not per flow: per critical flow, which in a normal company is three to five. Each card answers six questions and nothing else:

  • What this flow does and what happens when it is stopped. Two lines of business language, not technical. "Creates the delivery note in the ERP when a web order comes in. If it is stopped, orders arrive but nothing gets picked."
  • How to check in thirty seconds whether it is alive. The exact link to the execution history, and what to look at there.
  • The three causes that have already happened, and their fix. Expired credential, provider quota used up, field arriving empty. With the specific click, not with the theory.
  • How to stop it completely. The button, the switch, the place. Stopping is a legitimate action and very often the right one: a dead flow does less damage than a flow making bad decisions.
  • What happens in the meantime (the manual fallback, next section).
  • Who it escalates to if none of this works, with a name and a phone number.

The runbook is a close cousin of the flow documentation, but it is not the same thing and the two are worth keeping apart: documenting your automations exists so somebody can maintain them and change them calmly; the runbook exists for the fifteen minutes after an outage. One gets read sitting down, the other standing up. Try to make one document do both and you end up with twelve pages nobody opens at 11:40 at night.

The manual fallback: invoicing by hand is not a failure, it is the plan

This is the piece most people skip, and the one that saves the most money. While the automation is down, the business keeps going. Orders keep coming in, customers keep writing and invoices keep coming due. The question is not "when do we fix it?", it is "how do we keep operating until it is fixed?".

There is a cultural reflex that gets in the way here: going back to the manual process feels like a defeat, like admitting the automation was never worth it. It is the other way around. A serious system has a degraded mode; the one without it is not more automated, it is more fragile. For each critical flow, write down the by-hand version of the process: who runs it, where they log what they are doing, and the maximum volume it holds before somebody has to start saying no.

That last part is the one almost nobody writes and the one that decides everything else: the manual fallback has a ceiling. Twelve orders a day can be keyed in by hand; four hundred cannot. Knowing your number in advance changes the conversation during the outage, because it turns "we can hold" into "we can hold until two in the afternoon, after that we stop taking web orders". And the parallel log of what got done by hand is not bureaucracy: it is what makes the later reconciliation possible without duplicating anything, which is a job with its own mechanics and is worked out in recovering the data lost in a failed automation.

The three-line postmortem: why the same outage comes back in March without it

The incident ends, everyone breathes out and nobody writes anything down. Three months later the same thing happens and the person who fixed it the first time no longer remembers how. That is the real cost of skipping the postmortem, and it does not get paid at the time: it gets paid in the repeat.

You do not need the ten-page document of a company with thousands of people. Google's SRE book —where much of this practice was born— insists on two things that do scale downward: that it be blameless, focused on why the system allowed the failure and not on who caused it, and that you can start with a very basic procedure and refine it over time. At your scale that is three lines, written the same day, wherever the runbook lives:

  1. What happened and since when. With the real start time, not the detection time. The gap between the two is, almost always, the most useful finding of the whole exercise.
  2. What fixed it. The specific action. This goes straight into the known-causes section of the runbook, and it is what makes the second time take ten minutes instead of three hours.
  3. What we changed so it does not come back. One single thing, with an owner and a date. If there is nothing, write that down too: "we accept this may happen again" is a valid, conscious decision; "we will see" is not.

When those three lines pile up over a year, you stop having anecdotes and start having a map: which flow goes down most, which provider has cost you the most hours, which fix you have applied six times without ever resolving the cause. That map is what turns reactive firefighting into real maintenance, the kind that gets ahead of things.

What to set up this week

All of the above fits in one afternoon if you do it in this order and do not try to make it perfect: the three-severity table with your processes by name; the calendar with one name per week and a channel that makes noise; the one-page runbook for your two most critical flows; the manual version of those two processes with the ceiling written down; and the habit of the three lines after every outage. None of this needs a new tool. It needs somebody to decide it cold and leave it written where it will be found.

The honest boundary: this is written for whoever runs a handful of flows and wants to sleep at night. When what sits underneath is dozens of processes, several teams and agents making their own decisions, incident response stops being a card and becomes a function with shifts, service level agreements and its own escalation mechanics: that is AI agent incident management. And if what you want is the severities, the on-call, the runbooks and the manual fallback set up on your current flows and operated by someone who is not you, that is operations automation: day two, done, instead of an intention written on a napkin.

Frequently asked questions

One specific person, with a name and an assigned week, decided before anything happens — not "the team" and not "whoever sees it". In a small company, realistic on-call has four pieces: one name per week on a shared calendar, an alert channel that actually interrupts —a call or a text, because neither email nor a muted Slack wakes anyone up—, a single backup who gets paged if the first person does not answer within fifteen minutes, and an explicit rule for compensating out-of-hours work. Plus one honest decision before you build anything: if your critical processes only have consequences during business hours, the right move is to have no night on-call and say so out loud, instead of sustaining an informal rotation nobody ever agreed to.

By the damage done for every hour that passes, not by how broken the system is. Three levels are more than enough: serious means money or a customer commitment is being lost right now —orders are not going out, nothing is getting charged, nobody is getting answered— and the on-call person gets called whatever the hour; annoying means the process half works or has a workaround, and it gets picked up first thing the next business day; everything else goes into the week's maintenance queue. PagerDuty warns that six or seven levels nobody can tell apart produce classification paralysis: three to five is the sweet spot, and with no on-call team three is plenty. Write the table with your processes by name —"web orders", "invoicing"— because a severity level that has to be interpreted at 11:40 at night is useless.

One page per critical flow —three to five in a normal company, not all of them— with six answers: what the flow does and what happens when it is stopped, in business language; how to check in thirty seconds whether it is alive, with the exact link; the three causes that have already happened and their specific fix; how to stop it completely, because stopping is usually the right action; what happens in the meantime, meaning the manual fallback; and who it escalates to if nothing works, with a name and a phone number. Do not confuse it with the flow documentation: documentation gets read sitting down so somebody can maintain the flow, the runbook gets read standing up in the fifteen minutes after an outage. And it is only finished when somebody else has run it without calling you.

Free AI Impact Plan

The guide is generic. Your plan isn't.

Tell us about your company and we'll ship back a diagnosis with priorities, numbers and what to implement first. No sales call, no charge.

Who responds when an automation goes down: on-call, severities and runbooks for a small company · Implementa