Skip to content
Implementa.
AI AgentsOpinion··7 min

LLM vs machine learning: when a classic model beats generative AI

LLM vs machine learning isn't a fashion contest: it's a question about the shape of your data. If it's a table with history —sales, defaults, churn—, a classic model is usually cheaper, more stable and easier to explain. If it's free text, the LLM wins. And in production, the two usually win together, each in its own lane.

Senior AI Operations Implementer

AI Operations Pod

The request arrives full of enthusiasm: “we want an AI agent that forecasts demand.” Attached: a spreadsheet with three years of weekly sales by SKU. The unspoken plan is to paste that table into a language model and ask it to guess next month. It'll work in the demo. It'll give you a different number every time you ask. And nobody will be able to explain why it ordered 400 units instead of 250.

The thesis in one line: half the problems getting budgeted today as “generative AI projects” are better solved by a plain old machine learning model trained on your own history. Not out of technical nostalgia: it's cheaper per prediction, it gives the same answer to the same input, and you can explain it to an auditor. The LLM wins when the data is language. Choosing well isn't about what's trending, it's about the shape of what you want to predict.

LLM vs machine learning: the question is wrong until you look at the data

A language model is an extraordinary machine for reading, summarizing, drafting and reasoning over text with no fixed structure. A classic model —a gradient-boosted tree, a regression, a time-series model— is an extraordinary machine for finding patterns in columns: thousands of rows with the same variables and a known outcome. Different tools for different data shapes, and most architecture mistakes come from using the trendy one on the wrong shape.

If what you have is…Usually winsWhy
A table with history and a known outcome (sales, defaults, churn)Classic machine learningIt learns from your own cases, it's cheap to run and it's stable
Free text: emails, contracts, tickets, call notesLLMIt understands language without someone defining every column first
Documents with chaotic layoutsLLM (often alongside OCR)It reads variable structure; the classic model needs a template
A rule that fits in one sentenceNeither: a business ruleThe cheapest, most explainable option is to train nothing

This isn't a consultant's hunch. In 2022, researchers from Inria and Sorbonne Université compared tree-based models with deep networks on 45 medium-sized tabular datasets (on the order of 10,000 rows) and found trees were still state of the art on that ground (Grinsztajn, Oyallon and Varoquaux, NeurIPS 2022). That study didn't test LLMs, but its lesson travels: on tabular data, the most sophisticated model isn't the one that predicts best.

Three jobs where the classic model wins outright

Demand forecasting

That's the opening scene. In the M5 forecasting competition —42,840 series of Walmart daily unit sales—, most of the top-ranked solutions relied on LightGBM, a gradient-boosted tree model (Makridakis, Spiliotis and Assimakopoulos, International Journal of Forecasting, 2022). No winner asked a chatbot to read the spreadsheet. There are now foundation models built specifically for time series, and they're worth testing as a baseline; but they're a different family, not a text assistant you paste a spreadsheet into.

Scoring and case classification

Probability of default, of a customer churning, of a lead converting, of an invoice being paid late. All of that is classification over columns, the natural habitat of tree-based models. Trained on your closed cases, they give you a probability, the weight of each variable behind it, and the same answer tomorrow for the same customer.

Anomaly detection

A payment that doesn't fit a vendor's pattern, a usage spike, an order with an impossible margin. Running millions of transactions through an LLM is slow and expensive. A statistical or tree model does it in milliseconds per row and leaves the LLM, at most, the job of writing the alert for whoever has to decide.

Where the LLM wins (by a mile)

The other side is just as clear. A classic model can't read a complaint email, can't tell that “the usual, minus the big box” is an order, can't pull the penalty clause out of a forty-page contract. There, the LLM isn't competing: it's the only sensible option. It also wins when you have no history —a new process with no closed cases to learn from— and when the output you need is language: a reply, a summary, a draft.

That's why picking a model is never about “which is best” but about which task. It's the same logic we apply inside an agent system when deciding which model to use in an agent based on task, latency and cost per case: whatever tops the leaderboard is rarely what fits each step.

The pattern that ships: the LLM turns text into columns, the classic model decides

The grown-up version of this debate isn't picking a side. It's chaining. The LLM reads the unstructured stuff —the customer email, the scanned delivery note, the sales rep's notes— and turns it into fields: reason, amount, urgency, product. Those fields feed a classic model that scores, forecasts or flags. And a person reviews whatever falls outside the corridor.

Each piece does what it does best, and the system is auditable step by step: you know what the LLM read, which columns it produced and which variable weighed most in the decision. We saw the same hybrid when comparing classic OCR and multimodal models for reading documents: in production the combination wins, not a technical tie. And where a person steps in isn't improvised; it's designed with human-in-the-loop criteria for AI automation.

When the decision affects a person, explaining isn't optional

There's a non-technical argument that often settles this on its own. If your model scores people —credit, hiring, pricing, access to a service—, you'll have to explain why. In the US, a creditor that denies credit must give the applicant the specific principal reasons under the Equal Credit Opportunity Act and Regulation B, and the CFPB put it in writing in 2022: using a complex algorithm doesn't excuse you from that duty (CFPB, Circular 2022-03). If you also operate in Europe, the EU Court of Justice ruled in December 2023 that calculating a credit score can itself be an automated decision under the GDPR when a third party relies on it decisively (CJEU, case C-634/21, SCHUFA, press release).

A tree model lets you point to which variables moved each case's score. An LLM can generate an explanation that sounds reasonable, but that's not the same as the reason it decided. For any decision with consequences for a person, that gap is what holds up or sinks your system in front of a regulator. And every decision should be logged with its why: that's exactly what AI decision traceability solves.

When NOT to build a classic model

Going against the hype doesn't mean going against it every time. A classic model is a bad idea when:

  • You have no history with outcomes. No closed cases, nothing to learn from. An LLM with good instructions, or a rule, gives you something useful from day one; the classic model gives you nothing.
  • You have very few rows. With a few dozen examples, any pattern the model finds is noise in a nice suit.
  • The process changes every month. If the business rules move faster than you accumulate data, the model will always be navigating with last quarter's map.
  • A rule solves it. “If the invoice is over X and the vendor is new, review it” needs no training. It's the cheapest, fastest and most explainable of the three options.

Four questions before you ask for a quote

  1. Does what I want to predict fit in one column with a known answer in my history?
  2. How many closed cases do I have, and how much do today's look like those from two years ago?
  3. Will someone have to explain each decision to a customer, an auditor or a regulator?
  4. Is my input a table, text, or both?

If you answer “yes, lots, yes, table”, the project isn't generative AI, and whoever sells it to you as such is charging you for the buzzword. If you answer “no, few, no, text”, the LLM is your tool. And if it's a bit of everything —the usual—, what you need is an architecture that puts each model in its lane. To sort out where to start, use the guide on which processes to automate with AI; to build the whole thing, with data, models and supervision, there's our enterprise AI infrastructure service.

You don't pick AI by the headline. You pick it by the shape of the data. And sometimes the most modern answer is a ten-year-old model, trained properly on your own numbers.

Shall we get it shipping?

If this resonated, 30-minute conversation with no commitment. We tell you what fits, what doesn't and the approximate price.

See cases
LLM vs machine learning: when a classic model beats generative AI · Implementa