Why your agent "doesn't remember anything" when you've loaded it with every document you own
It is the number one complaint of week two: you uploaded the whole manual, the policies, the ticket history, and the agent still asks about things you told it yesterday. The immediate conclusion — "it needs memory" — is correct and completely useless, because memory is the word the market uses for three different things that live in different places, cost different money and break in different ways.
The most expensive confusion in this sector is this one: memory and RAG are not the same thing. RAG is the agent knowing how to look things up in your documentation. Memory is the agent knowing who you are the next time you talk to it. You can have flawless RAG — it pulls the exact paragraph out of the manual in 200 milliseconds — and an agent that still doesn't remember this customer has already complained twice about the same order. Different problems with different solutions, and building the second one with the first one's tooling is exactly why half of these projects get stuck right here.
The three things you throw in the same bucket
Before you buy anything, separate them. An agent in production usually needs all three, but they get bought, paid for and audited separately.
1. The context of the conversation in progress (short-term memory)
It is what the model has in front of it right now: everything said so far in this thread. It is not a database, it is the context window — the model's workspace — and it empties when the conversation ends. It is free in the sense that there is nothing to build, and extremely expensive in the sense that you pay for it by the token on every turn: the longer the thread, the pricier each next message. When somebody tells you their agent "remembers everything inside the conversation", they are describing the default behaviour, not a capability anybody built.
The symptom that this is what you're missing: the agent contradicts itself inside the same thread, or loses the plot in long conversations. You fix it with context management — summarising old turns — not with a vector database.
2. The knowledge it can look up (this one really is RAG)
Your documents, indexed so the agent can find the relevant fragment and answer with it in front of it. Manuals, policies, contracts, procedures, catalogue, resolved tickets. This is knowledge belonging to the organisation, the same for everybody, and its problem isn't remembering: it's finding and staying current. It lives in an index — vector, lexical or both — and its main cost isn't storage, it's maintenance: an index with the superseded procedure still sitting inside it competes silently with the live one.
The symptom that this is what you're missing: the agent answers confidently with things that are written nowhere, or cites the wrong document. You fix that with retrieval, not with memory. How it gets built and how it gets evaluated is in how to train an AI agent; keeping it alive is a continuous job with a name of its own, keeping the AI knowledge current.
3. The facts it has to remember between sessions (long-term memory)
This is the layer almost nobody builds, and the one you were actually missing. It is not documentation: it is concrete facts about a person, an account or a case, learned during the work, that have to survive the end of the conversation. "This customer invoices under a different legal entity." "This supplier will not accept delivery notes by email." "This user already turned this proposal down in March, and said why."
It is told apart from the other two by three properties: it is specific to one entity (it is no good for everybody else), it writes itself during the operation (nobody uploads a document containing it), and it expires (a preference changes, a fact stops being true). That third property is what turns long-term memory into a governance problem and not just an engineering one, and it is what the second half of this guide is about.
Where each one lives and what it costs
| Layer | Where it lives | Who writes it | What breaks it | Dominant cost |
|---|---|---|---|---|
| Conversation context | The model's context window | The conversation itself | Long threads: it overflows or it gets expensive | Tokens per turn |
| Look-up knowledge (RAG) | An index over your documents | You, when you publish the document | Stale and duplicate content nobody flagged | Index maintenance |
| Long-term memory | Your own store, outside the model | The agent, during the operation | Expired facts nobody retires | Governance: review and deletion |
Look at the right-hand column, because it is the one that never shows up in the budget. The first two layers have a cost you can see on an invoice. The third has a cost you can only see in a meeting: somebody has to decide what gets remembered, for how long, and who is allowed to delete it. If that somebody doesn't exist, long-term memory isn't a capability: it's a liability that grows on its own.
The thirty-second test for working out which one you are missing
Take the specific case that made you think you needed memory and ask it three questions. The answer tells you what to buy, and it stops you paying for a vector database to fix a conversation summary.
- Is the fact you are missing written down in a document of yours? If it is, this isn't memory: it's retrieval. Your problem is in the index or in the document, not in the agent.
- Is the fact good for all your customers, or only for this one? If it works for everybody, it's knowledge. If it belongs to this customer and nobody else, it's long-term memory.
- Would the fact stop being true within a year? If the answer is yes, you need the forgetting layer before you need the remembering one. Storing with no expiry plan is stockpiling future errors with an activation date on them.
Forgetting is harder than remembering
This is the part almost nobody writes down, and the part that costs the most. Building memory is a solved problem: you store the fact, you retrieve it when it's due. Retiring it, on the other hand, isn't one operation: it's several, and one of them failing is enough for the fact to stay alive.
The best public example of how tangled this gets comes from OpenAI's own documentation for ChatGPT. Its help centre says, in as many words, that to fully delete something ChatGPT may know about you "you'll need to delete each source where it appears, including past chats, archived chats, files, memory summary, and disconnect any connected apps that may contain this information". And it adds two warnings worth their weight in gold to anyone designing this inside a company: deleting a chat does not delete the memories that came out of it, and if you turn memory off and then back on, the system may create memories again from the chats still sitting in your history. Source: Memory FAQ, OpenAI Help Center, accessed 10 September 2026.
This is not an OpenAI defect: it is the shape of the problem. The moment a fact gets copied to more than one place — the memory store, the index, the conversation log, the destination system the agent wrote into — deletion stops being a button and becomes a procedure with a list of places attached. If nobody has written that list, you cannot promise you delete anything.
What actually puts you on the hook for being able to delete (and what nobody tells you)
Three anchors, and it pays not to mix them up. The first one binds you at home: under the CCPA as amended by the CPRA, a California consumer can ask that their personal information be deleted — and the statute writes the propagation problem into the law itself. Section 1798.105(c)(1) requires the business to delete the information from its own records, notify its service providers and contractors to delete it from theirs, and notify every third party it sold or shared the data with to delete it too. That is the list of places from the paragraph above, made into an obligation. Source: California Consumer Privacy Act, consolidated statute text, California Privacy Protection Agency, in force 1 January 2026. The second only reaches you if you process personal data of people in the EU: there the GDPR right to erasure covers whatever personal data you handle, and an agent's memory contains it by design — names, preferences, incidents, decisions. On top of that, the European Data Protection Board, in its Opinion 28/2024 (17 December 2024), sets a high bar for calling a model trained on personal data anonymous, and reminds everyone that in the face of unlawful processing the authorities can order corrective measures going as far as erasing the dataset. That opinion is arguing about the model; your memory store is a far simpler and far more clearly identifiable case, so what is debatable there is not debatable here.
The third binds nobody outside its own jurisdiction and is still the most operational thing anybody has published on this: in its guidance on agentic artificial intelligence, Spain's data protection authority lists memory compartmentalisation and traceability of the data lifecycle among the measures it expects. Compartmentalising means one customer's memory does not leak into another customer's conversation, and that it can be pointed at and retired in parts. It is the difference between a warehouse with labels and a messy drawer. Worth reading even where it does not apply to you. Source: Guidance on agentic artificial intelligence, AEPD, version 1.2, February 2026.
What none of the three tells you: how long your agent should remember that this customer prefers to be called in the afternoon. That is in no rulebook. It is a policy of yours, and if you do not write it down, the default policy is "forever", which is the worst one available.
The one-page sheet: what it remembers, where, and until when
Everything above fits in a table you review once a quarter. You do not need more than that, and without it there is no conversation to be had with whoever owns privacy in your company.
| What gets remembered | Layer | Expiry | Where it has to be deleted | Who can delete it |
|---|---|---|---|---|
| Customer contact preference | Long-term memory | 24 months with no activity | Store + copy in the CRM | Account owner |
| Open incident and its history | Long-term memory | Closure + legal retention | Store + ticket + log | Head of support |
| Current internal procedure | RAG | When the new version is published | Index (reindex) | Document owner |
| Summary of the thread in progress | Context | End of the conversation | Does not persist | — |
- One row per type of data, not per system. If a row says "customer data", it is badly written: split it until every row has a different expiry.
- The expiry, in units. "When we no longer need it" is not an expiry. "24 months with no activity" is, because a script can enforce it.
- The where-to-delete column is the important one. It is the list of places we were talking about: if it has a single entry, it is probably incomplete.
- One name that can delete. A person with a real button, tested at least once. A deletion procedure nobody has ever run is not a procedure.
- Quarterly review with the table in front of you. What piles up unreviewed stops being memory and becomes sediment.
We build the memory layer with this table written before the first line of code: it is part of the AI infrastructure the AI employees we put to work run on, inside the general logic of building an AI agent that survives production. We don't sell infinite memory. We charge for your agent remembering what it should, for as long as it should, and for the day it has to be deleted being a day it actually gets deleted.