Observability

What it includes

• Log every run: input, steps, output, cost and latency.
• Trace the agent’s decisions so you can reconstruct why it did what it did.
• Alerts when something drifts from normal (cost, errors, latency).
• Dashboards per process and per agent for the operational owner, not just engineering.

When you need it

• An agent failed at a customer and you can’t reconstruct what happened.
• AI cost is rising and nobody knows which process is driving it.
• You learn about problems from the customer, not from your dashboards.

How it’s measured

• Time to detect an incident (how long until the alarm fires).
• Share of runs traceable end-to-end.
• Cost per process visible in near real time.

Common mistakes

• Logging only the final output and not the steps: when it fails, there’s no way to know why.
• Dashboards only engineering understands while the process owner flies blind.
• Alerting on everything until nobody pays attention to any alert.

FAQ

Aren’t observability and evaluation the same thing?

No. Evaluation measures quality against known cases, usually before shipping. Observability watches what happens in production with real traffic. You need both: one prevents, the other detects.

When do you set up observability?

Before scaling. A pilot can live without it; a function with several agents in production and no observability is a function you don’t control.

Related capabilities