AI Engineering

Persistent Memory for LLM Agents: Patterns That Work

By nimblabs editorial··8 min read
Diagram of memory patterns feeding an LLM agent: file, retrieval, and scratchpad

Key takeaways

  • LLM agents are stateless — 'memory' is something you build around them.
  • Four core patterns: curated memory file, retrieval, scratchpad/state, and summaries.
  • Match the pattern to the data: durable facts vs task-specific vs evolving session state.
  • Curate ruthlessly — memory is about relevance, not hoarding everything.
  • Stale memory is dangerous; keep it current or the agent acts on wrong facts.

Why agents need external memory

An LLM has no memory of its own between calls — each request is independent, bounded by the context window. Anything you want it to 'remember' has to be re-supplied from outside.

So 'agent memory' isn't a model feature; it's an architecture you design around the model. The patterns below are the building blocks.

Pattern 1: a curated memory file

For durable facts — architecture, conventions, key decisions, constraints — a maintained file (like CLAUDE.md) that the agent reads each session is the simplest, most reliable memory.

It's high-signal and human-editable. The discipline is keeping it compact and current rather than letting it sprawl.

Pattern 2: retrieval (RAG)

For large or changing knowledge that won't fit in the window, store it externally and retrieve only the relevant pieces per task. This keeps the context focused and the knowledge base unbounded.

Retrieval shines for documentation and codebases; the risk is pulling irrelevant chunks, so quality of retrieval matters as much as storage.

Pattern 3: scratchpad and state

For an ongoing task, let the agent keep a working scratchpad — a running list of what it's done, decided, and still needs to do. This carries state across steps within a job.

Pair it with summarization: when a session gets long, compress the history into a short state summary and re-anchor, so the window stays usable.

Choosing and maintaining

Match the pattern to the data: curated file for stable facts, retrieval for big/changing knowledge, scratchpad+summaries for evolving session state. Most real agents combine all three.

Whatever you use, keep it curated and current. Memory's value is relevance; stale or bloated memory misleads the agent and wastes context.

Frequently asked questions

Do LLM agents have memory?+

Not on their own — they're stateless between calls. Memory is engineered around them with files, retrieval, scratchpads, and summaries.

What's the simplest form of agent memory?+

A curated project memory file the agent reads each session, holding durable facts like architecture, conventions, and decisions.

When should I use retrieval instead of a memory file?+

When the knowledge is too large or changes too often to fit in the context window — store it externally and retrieve only what each task needs.

How do I stop agent memory from becoming a problem?+

Keep it curated and current. Relevance beats volume; stale or bloated memory misleads the agent and wastes context budget.

Keep reading

Why Your AI Coding Agent Keeps Losing Context (and How to Fix It)

AI coding assistants forget your architecture and decisions between sessions. Here's why context windows fail and the workflow that keeps your project memory current.

Context Engineering for AI Coding Agents

Context engineering is the practice of feeding AI coding agents the right information at the right time. Here are the core principles that make agents reliable.