Agent memory is how an AI agent retains information, both within a single task and across sessions. Models are stateless by default: without memory, every request starts blank. Memory systems give agents working state for the current task and persistent knowledge of past interactions, preferences and outcomes.
Key takeaways
- Short-term memory is the context window: everything currently in the prompt.
- Long-term memory is external storage, retrieved when relevant.
- Long-term memory is usually implemented with a database plus retrieval.
- Deciding what to write to memory is harder than storing it.
- Poor memory design causes stale, contradictory or leaked information.
Agent memory explained in simple terms
Short-term memory is what you hold in your head during a conversation. Long-term memory is your notebook: you cannot remember everything, so you write down what matters and look it up later. An agent needs both, and needs discipline about what goes in the notebook.
Types of memory
| Type | Holds | Typical implementation |
|---|---|---|
| Working (short-term) | Current task state, recent turns, tool results | The context window |
| Episodic | What happened in past sessions | Summaries stored and retrieved |
| Semantic | Facts about the user, company or domain | Key-value store or vector store |
| Procedural | How to perform recurring tasks | Instructions, playbooks, learned prompts |
| Scratchpad | Intermediate reasoning within a task | Temporary notes in context or a file |
Managing short-term memory
Context is finite, so long tasks need management:
- Summarise older turns into a compact recap.
- Trim tool outputs to what matters.
- Offload large artefacts to files and keep references.
- Re-inject the goal periodically so it does not drift out of focus.
Long-term memory: the write problem
Storing everything is useless; retrieval becomes noisy. Good systems decide deliberately what to remember:
| Good candidates | Poor candidates |
|---|---|
| Stable preferences (“always reply in Hindi”) | One-off details from a single task |
| Durable facts (“finance approvals go to Priya”) | Transient status that expires quickly |
| Corrections the user had to repeat | Information available from systems on demand |
| Outcomes of past attempts | Full transcripts |
Memory should also support updating and deleting, since facts change and users may ask for information to be removed.
A worked example
A support agent handles a returning customer.
- On session start, it retrieves stored facts: enterprise plan, prefers email, previously reported a sync issue.
- During the session, working memory holds the current ticket and tool results.
- The customer mentions a new billing contact. The agent writes that to long-term memory as a durable fact.
- It does not store the entire conversation, only a short summary and the new fact.
- Next session, the relevant facts are retrieved again, so the customer does not repeat themselves.
Design considerations
- Relevance: retrieve only memories that affect the current task.
- Privacy: avoid storing sensitive personal data; support deletion.
- Conflict handling: newer information should supersede older.
- Provenance: record where each memory came from.
- Scoping: separate per-user, per-team and global memory.
Limitations
- Retrieval can surface irrelevant or outdated memories.
- Stored errors persist and compound.
- Privacy and compliance obligations apply to stored personal data.
- More memory means longer prompts, raising cost and latency.