Solution · AI Operations
Your AI decides every day. Six months from now, can you explain one specific decision?
A log is not a trace. Having records scattered across the model, the tool and the destination system does not let you reconstruct why this request was denied, this price was applied or this case closed on its own. Decision traceability is the function that ties every decision to its input, its versions, its data, its policy and its approver —and keeps it retrievable when someone asks. We build it and we run it.
The problem
You have logs. What you do not have is the ability to reconstruct one specific decision end to end.
- Information about a single decision lives in four places —the model call, the retrieval engine, the tool that executed and the system where it landed— and none of them share an identifier, so reconstructing it is hours of manual work.
- You do not know which version decided: the prompt, the model, the business policy and the dataset have all changed since, and nobody recorded which was live at that moment.
- The record keeps the output but not the why: which documents it retrieved, which tools it called, what they returned, and which rule was applied to decide this way and not another.
- Human oversight leaves no usable trail: someone approved, but there is no record of what they saw on screen, what changed versus the proposal, or on what basis they decided.
- Retention is undecided: either everything is kept forever —personal data included— or it rotates after thirty days, and by the time the complaint arrives there is nothing left to show.
Cost of staying the same
Traceability is not optional paperwork: for high-risk systems, the EU AI Act requires automatic event logging across the lifecycle (Art. 12) and mandates keeping those logs for at least six months, both for the provider (Art. 19) and the deployer (Art. 26(6)) — which applies to you if you operate in or sell into the EU. But the regulator is the one who arrives last. First comes the customer disputing the decision, the rep who cannot understand why their deal was denied, and the team that wants to improve the system and cannot, because without a trace there is no diagnosis: you cannot tell a model failure from bad data, or a badly written policy from a legitimate exception. Without a trace, the only available answer is "the AI decided it", and that sentence holds up with nobody —not an auditor, not a customer, not your own board.
The solution
We build end-to-end decision tracing —identifier, versions, context, policy and oversight— and we run it as a continuous function
- 1We start by defining what counts as a "decision" in your business, the step almost nobody takes: not everything an agent does deserves a file. We mark the consequential decisions —those affecting a person, a payment, a contract or a commitment— and those get traced in detail; the rest gets sampled.
- 2We give every case a unique identifier and push it through everything: the model call, the information retrieval, every tool invoked and the final record in your ERP, CRM or ticketing. Without that thread, reconstructing a decision stays archaeology across four systems.
- 3We freeze the decision context: the live version of the model, the prompt and the business policy, the documents or records retrieved, the parameters used, the tools called and what they returned. The trace keeps not just what came out, but what it was decided on.
- 4We make human oversight leave a real trail: what the reviewer was shown, what the system proposed, what the person changed and why. That is the difference between "there is oversight" and being able to prove it, and it is what connects to human oversight of AI at scale.
- 5We set retention and access with judgment: how long each trace type is kept, what gets masked or swapped for an identifier so you do not hoard personal data without a legal basis, who can query it and with what access log. Keeping everything forever is not compliance: it is another risk.
- 6We leave it queryable and measured. A decision file you open by case ID and read in minutes, not a log dump; plus a coverage panel —what share of consequential decisions has a complete trace— and periodic drills: we pick a decision at random and reconstruct it, to find the gaps before someone else does.
What changes
What you stop losing
The disputed decision stops being archaeology across four systems: it opens by case ID, with its input, context, versions and approver in a single file.
Mechanism
For high-risk systems, the EU AI Act requires automatic event logging (Art. 12) and a six-month minimum retention (Art. 19 for the provider, Art. 26(6) for the deployer). Tracing stops being a pending project and becomes an obligation covered with evidence.
Regulation (EU) 2024/1689 (AI Act), Arts. 12, 19 and 26(6) — scope: European Union
Improving the system stops being blind: with the trace you can tell a model failure from bad data or a badly written policy, which is what decides where it is worth touching.
Mechanism
What we measure: coverage of consequential decisions with a complete trace, time to reconstruct a case end to end, % of reconstructions that fail on a gap, and age and volume of trace retained by type.
What we measure
Spec sheet
- Work it removes
- reconstructing an AI decision by hand, digging through logs in four systems that share no identifier, with no way to know which model, prompt or policy version was live
- Typical setup
- 4–8 weeks
- Input
- a consequential decision made by your AI —a denial, a price, a case closure, an action on a system— and everything around it
- Output
- a decision file retrievable by case ID: input, retrieved context, tools invoked, versions, policy applied, human oversight and outcome
- Works with
- Los frameworks de agentes que ya uses (LangChain, LlamaIndex, SDKs de proveedor, desarrollo propio)OpenTelemetry y tu plataforma de observabilidad o de trazas de LLMTu ERP / CRM / ticketing como sistema de destinoTu gestor de versiones de prompts, políticas y modelos
- Can connect to
- Your AI Act compliance program and your audit evidenceYour retention and personal data protection policyYour human oversight and approval workflowYour incident and continuous improvement process
- What we measure
- coverage of consequential decisions with a complete tracetime to reconstruct a case end to end% of reconstructions that fail on a gap in the traceage and volume of trace retained by decision type
- Good fit for
- companies with AI or agents making consequential decisions in production —credit, pricing, hiring, support, operations— that will have to explain them to a customer, an auditor or a regulator
- Not a fit for
- anyone only running writing assistants or internal pilots with no effect on third parties: operational observability is enough there; and it does not replace the legal judgment of what is defensible, which stays with your counsel
Frequently asked questions
No, and confusing the two is the expensive mistake. Monitoring AI in production answers an aggregate question in the present: is the system healthy today, what does it cost, where is it degrading? Traceability answers a singular question in the past: why did this decision come out this way, the one from March 14th, the one this customer is disputing? One watches the curve; the other opens the file. You need both, but the one that saves you in a dispute or an audit is the second — and it is the one almost nobody has built, because it requires deciding what gets stored, under what identifier and for how long.
Complying with the AI Act while running your AI is the framework: which systems you have, which risk category they fall into, what documentation and oversight each one requires. Traceability is one of the mechanisms that framework assumes and that is almost never built: the automatic event logging of Article 12 and its six-month minimum retention. You can have your compliance program current and still be unable to reconstruct one specific decision — which is exactly what will be asked of you. They fit well together, and if the program is already running, this is the missing piece.
It does, if you do it bluntly, which is why retention is part of the design and not a later adjustment. We trace in detail only consequential decisions and sample the rest; we swap personal data for identifiers wherever the trace stays reconstructable; we set periods by decision type —covering whatever legal minimum applies, without turning it into "forever"—; and we log who opens a file. Keeping everything indefinitely is not better compliance: it is accumulated risk and an accumulated bill. The rule is that the trace has to be enough to reconstruct, and no more than that.
Want it running in your business?
You’ve pinned the problem. We ship the fix and leave it measured.