← Full glossary
Explainable AI (XAI)
gobernanzaXAI · AI explainability
Explainable AI is the set of practices that let a person understand why an AI system made a decision or took an action, instead of accepting it as a black box.
Explainable AI answers the question 'why did it do that?'. With a model that only classifies, justifying the prediction is enough; with an agent that plans, calls tools and chains steps, you must be able to reconstruct the sequence: what it received, what it reasoned, which tool it called and with what result. In operations it underpins audit, error correction and the confidence of the team supervising the agent. It does not replace observability, which records what happened; explainability makes it understandable to a person.
How it differs from
- Agent observability
- Observability records the what; explainability supplies the understandable why.
FAQ
- Is explainable AI the same as agent observability?
- No. Observability records what the agent did; explainable AI makes that record understandable and defensible to a person or an auditor.
Related terms
Sources & further reading
- Red Hat — What is AgentOps? · 2026-10-02