Skip to content
Implementa.

Building an AI agent · Guide 13 of 13

Which model to use in an AI agent: the question is not which one, but how many and for what

Someone opens a leaderboard, points at the top spot and asks whether we should build the agent on that one. Fair question, wrong question: it assumes an agent uses one model, and the ones that survive production use several. The real decision is not which, it is the split — cheap for classifying and routing, expensive for deciding and for anything your customer reads — plus the layer that lets you swap them without rebuilding anything. Here are the three axes that actually decide, the order you walk them in, and what to write down so the decision does not live in one person's head.

"Which is the best model?" is the question of someone who has not shipped one yet

The conversation always starts the same way. Someone opens a leaderboard, points at the top spot and asks whether we should build the agent on that one. It is a fair question and it is the wrong one, because it assumes an agent uses one model. Agents that survive production do not use one: they use several. And the decision that actually matters is not which, it is the split.

The reason is architectural and boring. An agent does not do one task: it does a chain. It reads an email and works out what it is about. It searches your documentation and pulls three passages. It drafts a reply one of your customers will read. It decides whether that goes out on its own or through a person. Four steps, four different demands: sorting an email into six categories has been a solved problem for years; drafting what your customer reads is where your face is on the line. Paying for the most capable model on all four is putting your best lawyer on photocopying duty.

This is not our opinion. OpenAI's guide to building agents says it plainly: models have different strengths and trade-offs around task complexity, latency and cost, and not every task needs the smartest model. Their example is exactly the one above: a simple retrieval or intent classification can be handled by a smaller, faster model, while deciding whether to approve a refund benefits from a more capable one. And they recommend considering a variety of models for different tasks in the workflow rather than one for everything. Source: A practical guide to building agents, OpenAI, accessed 12 September 2026.

This guide is about making that decision on purpose: the three axes that actually decide, the order you walk them in, how the split works, and what layer you need so that changing model is not a rebuild. What it is not: a "best LLMs of 2026" table — that expires before we publish it — nor the bill for running AI in production, which is a different conversation with different numbers.

The three axes that actually decide

Strip out the noise and the choice rests on three questions. None of the three is answered by reading a model card: all three are answered by measuring in your own house.

AxisThe question it answersHow you measure it in your case
Accuracy on your taskHow many of my real cases does it get right?A set of 30-50 of your own cases with the correct answer written next to each
LatencyDoes it hold up in the channel the agent lives in?The 95th percentile of response time, never the average
Cost per resolved caseWhat does it cost to resolve one end to end?Total cost of the whole flow divided by resolved cases, not price per million tokens

Accuracy is the axis most people skip, because it is the one that takes work. "Accuracy" is not a general grade: it is the percentage of YOUR cases that come out right. Getting it means doing the boring part — thirty or fifty real cases, actual emails with their typos and their odd attachments, with the correct answer written next to each. Without that set you are not choosing a model: you are holding an opinion about models, which is a different thing and does not survive a meeting.

Latency is not an absolute number: it is a number against a channel. A chat on your website has a budget of seconds because there is a person watching the screen. A nightly job reconciling invoices has all night. The same model is fast in one and slow in the other, so "is it fast?" means nothing until you say where. Anthropic frames it as what it is — a trade: agentic systems often trade latency and cost for better task performance, and you should consider when that trade makes sense. Source: Building effective agents, Anthropic, 19 December 2024, accessed 12 September 2026.

Price per million tokens is a list price, not your bill. A cheap model that needs three attempts, leaves half the fields empty and ends up escalating to a person costs you more than an expensive one that gets it right first time. The correct unit is the resolved case: total cost of the flow — every call, every retry, including the ones from the step that failed — divided by the cases that came out right without anyone touching them.

The public leaderboard knows nothing about your work

With the three axes on the table it is clear why the leaderboard does not decide. A public ranking measures a standardised exam: quiz questions, toy coding problems, logic puzzles. Your task is not that. Your task is your customers' emails, your internal jargon, your badly scanned PDFs and your returns policy, and none of those four appear in any table. We made the case at the time: LLM benchmarks do not predict your result.

What a public ranking is good for is the opposite of what people do with it: it is for ruling out, not for choosing. It tells you which models are in the conversation and which fell behind two generations ago, and that saves you from evaluating fifteen candidates. From there, the shortlist — two, three — gets ordered by your own case set. That order rarely matches the leaderboard, and when it does, it is a coincidence you cannot bank on next time.

The split: cheap to classify, expensive to decide

The pattern has a name and it is documented. Anthropic calls it routing: a first step classifies the input and directs it to the specialised follow-up task it belongs to. Its value, they explain, is separation of concerns and the ability to write more specialised prompts, because without that split, optimising for one kind of input hurts performance on the others. And one of the examples they give is literally the cost split: routing easy, common questions to smaller, cost-efficient models, and hard or unusual ones to more capable models. Source: Building effective agents, Anthropic, 19 December 2024, accessed 12 September 2026.

Translated into the steps of a typical agent, the split usually lands like this. The right-hand column is a starting point, not a law:

Agent stepWhat it really demandsWhere it usually lands
Classify the input and route itConsistency and speed over a closed set of categoriesSmall model
Extract fields from a documentStable format; errors are caught by validationSmall model with an output contract
Search and summarise your documentsFaithfulness to the source; retrieval quality dominatesMid-tier model
Decide on money, people or regulated dataJudgement and nuance; errors are paid in euros or reputationMost capable model
Draft what a customer will readTone, precision and zero inventionMost capable model

The only thing that can confirm that table is your own case set. There are classification jobs with twenty overlapping categories where the small model sinks, and drafting jobs so narrow — an acknowledgement with three variables — that the small one is overkill. That is why the split is decided by measuring, and revisited when the model catalogue changes or your volume does.

One nuance that saves grief: routing only pays off when the categories are genuinely distinct and classification can be handled accurately. That is the condition Anthropic attaches to this pattern, and it is the one that fails when someone drops a router in where none was needed. If the classifier gets it wrong you have not saved anything: you have put a new point of failure in front of everything else, and a silent one at that, because a misrouted input does not throw an error — it returns a confident answer of the wrong kind.

The right order: start expensive and come down until it shows

With the split clear, the question is how. OpenAI's guide proposes a recipe that runs against everyone's instinct: build the prototype with the most capable model for every task to establish a performance baseline, and from there try swapping in smaller models to see whether they still achieve acceptable results. Their principles, in this order: set up evals to establish the baseline, meet your accuracy target with the best models available, and only then optimise for cost and latency by replacing larger models with smaller ones where possible. Source: A practical guide to building agents, OpenAI, accessed 12 September 2026.

  1. Build the case set before you touch the agent. Thirty or fifty, real, with the correct answer next to each. This is the step everyone skips and the one that decides whether the rest is worth anything.
  2. Build the whole agent with the most capable model at every step. You are not optimising yet: you are finding out whether your accuracy target is reachable at all.
  3. Measure against the set. If you fall short here, the problem is not the model: it is the prompt, your data, the retrieval or the task — and dropping to a smaller model will only hide it.
  4. Once you hit the target, come down one step at a time and re-run the set. One change, one measurement. Two changes at once and you no longer know which one it was.
  5. Stop at the rung before the one that breaks. And write down which step uses which model and at what measured accuracy, because in three months nobody will remember.

The reason not to do it the other way round is diagnostic, not purist. If you start with the cheap model and the agent does not work, you have four suspects and no way to separate them: it could be the model, the prompt, your data, or a task that was never properly defined. Starting from the top, when something breaks on the way down you know exactly what broke it, because you only changed one thing.

The layer that lets you change models without touching the agent

Everything above has an expiry date. The model catalogue turns over every few months, prices move and providers deprecate versions. If today's decision is written inside the agent — the model name repeated in seven places in the code — in six months you will not be able to redo it, and you will end up with the problem we described in how to change model without breaking your automations: the flow does not break with an error, it breaks with a different format and a different tone, quietly.

The layer that prevents it is five pieces, and none of them is sophisticated:

  • One single function that calls the model. Every call goes through it. If there are seven places in the code with a model name inside, you already have debt.
  • The model identifier, in configuration. One file with the model for each step. Changing model has to be changing a line, not opening the agent.
  • A validated output contract. Before the result touches any system, you check it has the agreed shape. That is what turns a silent format change into a visible error.
  • The case set, runnable with one command. If trying a new model costs half a morning of manual work, nobody will try one.
  • A log of which model handled which case. Without it you cannot compare before and after a change, or explain why last week it worked better.

This is not an optional optimisation you defer to phase two. It is the difference between choosing a model today and being able to choose again in six months. When that layer does not exist, the model decision is made once and inherited forever, which is exactly the wrong shape for a system living in a market that moves every quarter.

The split sheet: one page, with an owner and a date

The ending is the same as in an AI agent's instructions: if the decision is not written down, it does not exist. One page is enough, and it has to answer these five things.

  • Which model each step uses, with one line of why and the accuracy measured against the set on the day it was decided.
  • What was tried and rejected. The candidate that did not make it and the reason. Without this, somebody will test it again from scratch in four months.
  • Who signs it. A named person. A sheet with no owner never gets reviewed.
  • What triggers a review: a relevant new model appearing, your provider deprecating the one you use, volume changing by an order of magnitude, or measured accuracy dropping.
  • Next review. A date, even if nothing has happened. The decision expires on its own and nobody announces it.

If your agent is already in production and this page does not exist, start with the steps that touch money: which model decides a refund, which model drafts what your customer reads, and at what measured accuracy. The rest can wait a week.

We build that layer before the agent: isolated call, model in configuration, output contract and a runnable case set, as part of the AI infrastructure for enterprise that the AI employees are then built on. It is the part nobody shows in a demo and the only one that decides whether, a year from now, you can change model in an afternoon or have to rebuild the system. The full logic is in building an AI agent that survives production.

Frequently asked questions

The right question is not which, but how many and for what. An agent does not do one task: it does a chain — classify the input, search your documents, draft, decide whether it goes out on its own — and each step demands different things. OpenAI's guide to building agents says it explicitly: models have different strengths and trade-offs around task complexity, latency and cost, not every task needs the smartest model, and it is worth using a variety of models for different tasks in the workflow; their example is that a simple retrieval or intent classification can be handled by a smaller, faster model, while deciding whether to approve a refund benefits from a more capable one. So the operating answer is: a small one to classify and extract, a capable one for anything touching money and anything your customer reads, and a set of your own test cases that confirms the split. Source: A practical guide to building agents, OpenAI, accessed 12 September 2026.

No, and using a single one is the default decision that costs the most. The pattern that avoids it has a name and is documented: Anthropic calls it routing, a first step that classifies the input and directs it to the specialised follow-up task it belongs to. They explain it allows separation of concerns and more specialised prompts, because without that split optimising for one kind of input hurts performance on the others; and one of their examples is exactly the cost split — routing easy, common questions to smaller, cost-efficient models and hard or unusual ones to more capable models. The condition they attach matters: routing works well when the categories are genuinely distinct and classification can be handled accurately. If the classifier gets it wrong, you have added a new point of failure in front of everything else. Source: Building effective agents, Anthropic, 19 December 2024, accessed 12 September 2026.

They are useful for ruling out, not for choosing, and that is the opposite of how they get used. A leaderboard measures a standardised exam — quiz questions, toy coding problems, logic puzzles — and your task looks nothing like that: it is your customers' emails, your internal jargon, your badly scanned PDFs and your returns policy. What the ranking does tell you is which models are in the conversation and which fell behind two generations ago. From there, the shortlist of two or three candidates gets ordered by a set of thirty to fifty of your own cases with the correct answer written next to each, and that order rarely matches the leaderboard. If you cannot say what percentage of your cases each candidate resolves, you do not have a decision: you have a preference.

The capable one, then you come down. OpenAI's guide recommends building the prototype with the most capable model for every task to establish a performance baseline, and from there swapping in smaller models to see whether they still achieve acceptable results; its principles are to set up evals to establish that baseline, hit your accuracy target with the best models available, and only then optimise for cost and latency by replacing larger models with smaller ones where possible. The reason not to do it the other way round is diagnostic: if you start cheap and it does not work, you cannot tell whether the problem is the model, the prompt, your data or the task itself — and you never learn whether the target was reachable at all. Source: A practical guide to building agents, OpenAI, accessed 12 September 2026.

By isolating the model call in one place and treating the model name as configuration, not code. In practice it is five pieces: a single function every call goes through, the model identifier in a config file per step, an output contract validated before the result touches any system, the test set runnable with one command, and a log of which model handled which case so you can compare before and after. With that, changing model is changing one line and re-running the set; without it, it is a rebuild. This layer is not an optional optimisation: it is the difference between choosing a model today and being able to choose again in six months, when the catalogue has moved and today's decision is no longer the right one.

Free AI Impact Plan

The guide is generic. Your plan isn't.

Tell us about your company and we'll ship back a diagnosis with priorities, numbers and what to implement first. No sales call, no charge.

Which model to use in an AI agent: the question is not which one, but how many and for what · Implementa