An AI agent that browses the web: the last resort, not the first

An AI agent that browses the web is the last resort, not the first

An AI agent that browses the web does exactly what it sounds like: it opens a browser, looks at the screen, moves the cursor, types and clicks, just as a person would. And that is the trap, because it sounds like a universal solution — if a person can do it, the agent can do it. True, and irrelevant. A person can also copy a thousand rows by hand. Something being possible does not make it the sensible way to do it.

The thesis, plainly: the browser is plan D. Ahead of it come the API of the system on the other side, integration through a connector or an iPaaS, and — more often than anyone admits — a person spending ten minutes a day on something that does not justify automating anything. The browser only enters when all three are genuinely closed. The clean routes are covered in integrating AI with your systems: that guide rules out anything that is not a port you can plug into, and this one starts exactly where it stops, in the case it rules out.

The reason for all that caution is that this route does not fail because of the model. It fails because of things outside your control: somebody else's screen, somebody else's session, somebody else's rules. You can do everything right and wake up on a Tuesday with the process broken because someone moved a button.

The three cases where there is no other door

There are situations where the browser is not laziness: it is all that is left. They share one signal — it is not that the API does not exist yet, it is that it is never going to.

1. The supplier or large-customer portal. You are required to log into their extranet to upload delivery notes, download statements or confirm orders. There is no API, there will not be one, and your volume does not justify them building one for you. The commercial relationship runs one way and it is not yours.
2. Government. Filing portals, tender platforms, registries. Access here is designed for a person holding their certificate, and the automated path either does not exist or requires a formal integration that takes months to be granted. Careful with assuming that public means open: they are not the same thing.
3. The old internal system. That 2009 ERP, the warehouse desktop application, the program installed by a vendor that has since shut down. It has a database, but nobody will sign off on touching it; it has a screen, and the screen works. The API does not exist because the budget to build it does not exist, which is a perfectly valid way of not existing.

The counter-example matters as much as the cases. If the supplier does have an API and has not given it to you because it sits in a different price tier, that is not a technical problem: it is a negotiation. Building a browsing agent to avoid paying for the connector usually costs more in first-year maintenance than the connector itself, and it leaves you on the wrong side of their terms of use. Before writing a line of code, ask for the price. Sometimes the most profitable automation is an email to a salesperson.

What always breaks: five failures that are not yours to control

These five are not hypothetical risks that might turn up. They are the maintenance calendar of any agent living inside somebody else's browser. Anyone selling you this without mentioning them has not run it in production.

• The interface changes and nobody tells you. No versioning, no release note, no grace period. A redesign, a new required field or a different cookie banner, and the flow stops halfway. With a broken API you find out from an error code; here you find out because somebody asks why the invoices never arrived.
• The session expires and the second factor does not negotiate. Keeping an agent authenticated inside a third-party system is the boring problem that eats half the operating effort. If access requires a one-time code, an app on somebody's phone or a certificate on a card, there is a person in the loop by design. That is not an agent failure: it is the other side's security decision, and a legitimate one.
• The captcha and the terms of use. A captcha is not a technical speed bump to be steered around: it is the other party explicitly saying it does not want automated traffic. Getting past it usually breaks the terms of service and, depending on the context, rather more than the terms. When a captcha appears, the right conversation is not with engineering, it is with whoever signs the contract.
• The page it reads can also give it orders. A browsing agent cannot reliably separate the content you asked it to read from an instruction hidden inside that content. This is not a maturity problem about to be fixed: OpenAI stated in December 2025 that prompt injection, much like scams and social engineering, is unlikely ever to be fully solved (TechCrunch, 22 December 2025), and Anthropic published its browser-use robustness results with an explicit warning that no browser agent is immune (Anthropic, 24 November 2025). The practical consequence is architectural, not a matter of prompting: the agent can only do damage as far as its credentials reach.
• Cost and the clock are counted per step. Every step is a screenshot, a model decision and an action. A twelve-step task is twelve decisions with images, not one call. What an API resolves in a request measured in milliseconds takes minutes here, with consumption that grows with every retry.

Why the benchmark score does not predict yours

The public numbers in this category are better than many people think and worse than the demo suggests. On WebArena, the reference test bed, the best system tracked on the public leaderboard sits around 74% of tasks completed against the 78% a person manages (Steel.dev, WebArena leaderboard). OpenAI reported 87% on WebVoyager and 58.1% on WebArena for its computer-using agent in internal testing. Respectable figures for a hard problem.

The problem is what they measure. Those test beds run on replicas of public sites, down the good path, with no corporate sign-on, no second factor, no captcha and — this is the important part — no consequence. A wrong click in a benchmark adds a failure to a table. A wrong click on your supplier's portal confirms an eleven-thousand-euro order. There is no way to carry a percentage from the first to the second, so do not try: the only number that counts is yours, measured on your portal, with your real cases, over a couple of weeks in observation mode.

Translated into a decision: do not buy on the demo or the leaderboard. Ask for a trial on your screen, awkward cases included — the supplier who writes the invoice number with a hyphen, the order with two cancelled lines — and count how often it has to be rescued. That percentage is what determines whether the process saves you work or simply moves it somewhere else.

The four conditions for putting it into production

If the case is legitimate and the trial went reasonably, what is still missing is the part that separates an experiment from something that can be left running on its own. Four conditions, none optional: with three out of four, this does not ship.

1. Reversible actions. The agent only executes things that can be undone without calling anyone: downloading, reading, filling in a draft, saving without sending. Confirm, pay, sign and cancel stay outside the automatic scope from day one, however capable the agent is of clicking them.
2. A hard step limit. A cap on actions per task that stops and raises a flag when reached. Without it, an agent lost on a screen it does not understand retries until the budget is gone, and the first sign that anything was wrong arrives on the invoice.
3. A record of every action. A screenshot or log of what it saw and what it clicked, at each step, stored outside the session. This is not bureaucracy: it is the only thing that lets you reconstruct events when the supplier says an order was confirmed that nobody remembers confirming. No evidence, no audit; no audit, and this is not a system, it is a bet.
4. A human on anything irreversible. The step that commits money, sends something to a third party or changes a state that cannot be rolled back goes through a person who sees what is about to happen and approves it. With enough context to decide in ten seconds, not a dialog that gets clicked out of habit.

The last two conditions are not specific to this route: they are the same ones that govern what permissions to give an AI agent and how much autonomy to grant it. What changes here is the margin: when the agent acts on somebody else's screen using one of your employees' credentials, the mistake does not stay in the building.

The decision order, in an afternoon

Four questions, in this order. The first yes decides, and if you reach the fourth you already know what you are exposing yourself to.

1. Is there an API, and can they give it to you? If it exists, even as a paid option, ask for the price before building anything. A boring connector beats a brilliant agent that breaks with every redesign.
2. Is there an integration already built — a connector, an iPaaS, a scheduled export? The nightly export to a file that nobody shows off in a demo resolves more processes than it gets credit for, and it does not break.
3. How much human work is actually here? Ten minutes a day does not pay for maintaining a browsing agent. Measure it before deciding; sometimes the output of the exercise is not automating, and that is a perfectly good answer.
4. If you have got this far, are all four conditions met? Reversible, step cap, everything recorded, human on the irreversible. If one is missing, the process is not ready, and forcing it means taking on risk to save a bit of paperwork.

We come to this route reluctantly and use it when it is warranted, which is more often than we would like: half of Europe still invoices against portals with no API. When that is the case, we build it with all four conditions in place from day one and with maintenance budgeted, because what kills these projects is not building them, it is the third redesign. That is what sits underneath operations automation and enterprise AI infrastructure when the process has to coexist with systems already in production. The rest of the build decisions — permissions, autonomy, memory, channel — are in building an AI agent that survives production.