Memory for agents

Memory is what lets an agent keep a fact past the end of a conversation. Without it, every lesson the agent learns, a fix that worked, a hazard to avoid, a decision that was reversed, vanishes the moment the session ends and has to be discovered again from scratch the next time.

A model does not remember you

Begin with a fact that surprises people. A language model does not remember your last conversation. It cannot. To see why, you have to know what a model is between the moment it is built and the moment you use it.

A model is trained once, over a long and expensive process, and the result of that training is a huge set of numbers called its weights. The weights are what the model learned. When training finishes, the weights are frozen. From then on, every time you use the model, those same frozen weights are what runs. Using the model is called inference, and inference does not change the weights at all. The model that answers you this morning and the model that answers you tonight are byte for byte identical, no matter what happened in between.

This means the model has no place to store anything about your particular conversations. It did not learn from what you told it yesterday, because learning only happened during training, long before you arrived, and it is over. Each time you start a fresh session, you are talking to the exact same frozen model as everyone else, with no trace of your history baked in. In that sense a model is stateless: it carries no state from one use to the next.

Then how does it seem to remember within a chat

This raises an obvious objection. Within a single conversation, a model clearly remembers. You tell it your name at the top and it uses your name at the bottom. If the weights never change, where did that memory live?

The answer is that it was never stored in the model. It was in front of the model the whole time. Everything the model can see while it produces its next piece of text, the whole conversation so far, the question, the earlier answers, is called the context window. Each time the model responds, it is handed the entire conversation up to that point and reads all of it before answering. The model remembers your name at the bottom of the chat because your name is still sitting there in the text it is rereading, not because it retained anything.

Picture a person with no long-term memory, sitting at a desk with a single sheet of paper that holds the whole discussion. Each time they answer, they read the entire sheet from the top, respond, and add their reply to the sheet. They appear to follow the conversation perfectly. But they are relying entirely on the sheet, not on memory, and the proof is what happens when the sheet is taken away. When the session ends, the sheet is thrown out. The next session begins with a blank sheet and a person who has never met you.

Two kinds of memory, and why the short kind is not enough

So there are really two different things people mean by memory, and keeping them apart is the whole point of this page. The context window is short-term memory: everything the agent can see right now, this session, on the sheet in front of it. It is immediate and detailed, and it is also temporary. It lasts exactly as long as the session and not one moment longer.

Durable memory is the long-term kind: facts deliberately saved somewhere outside the model and outside any single session, so they can be brought back later. This is what the frozen model lacks and the context window cannot provide, because the context window is wiped when the session ends. If you want an agent to know next week what it learned today, that knowledge has to be written down somewhere that survives the end of the conversation.

The context window is a sheet of paper the agent reads and then throws away. Durable memory is a notebook it keeps. Only the notebook survives the end of a session.

There is a second reason the short-term kind is not enough, beyond its short life. The sheet has a fixed size. A context window can only hold so much text at once, and a large codebase or a long history of work is far too big to fit. You cannot solve the problem by simply putting everything on the sheet every time, both because the sheet is too small and because most of what happened is noise you do not want the agent rereading. Durable memory is not just longer lived than the context window. It is also selective, holding the few things worth keeping rather than everything that ever happened.

What is actually worth remembering

Because memory is selective, the question of what to save is the heart of the matter. Saving everything is the same as saving nothing, because a memory full of trivia is one you cannot find anything in. Good memory holds facts that are durable, that will still be true and still matter later, and that would be costly to rediscover. A few kinds stand out.

• A fix that worked. When a stubborn problem is finally solved, the solution is worth keeping, so the next time the same problem appears the agent reaches for the answer instead of rediscovering it.
• A hazard. When some part of the system turns out to be dangerous to touch, or fragile in a way that is not obvious from looking at it, that warning saves the next session from walking into the same trap.
• A revert. When a change was tried and then undone because it caused trouble, the fact that it was reversed, and why, keeps the agent from confidently trying the same thing again.
• A durable, code-anchored fact. A stable truth about how the system is built, tied to a specific place in the code, that will still hold sessions from now and would otherwise have to be worked out again each time.

Notice what these have in common. Each is a lesson that was expensive to learn the first time and cheap to reuse once written down. That is the exact profile of a fact worth remembering: hard to earn, easy to apply, and stable enough that it will not be stale by the time it is needed again. Passing details, the exact wording of a question, a step already completed, do not meet that bar and are better left to the short-term sheet.

How memory is stored and brought back

For durable memory to work, two separate actions are needed, and it helps to name them plainly: writing a fact down, and reading it back later. Writing is the act of saving a new fact into the store. Reading, usually called recall, is the act of pulling relevant saved facts back into the context window at the moment they are useful, so the model can see them on its sheet again.

The store itself is just a place outside the model that keeps the facts between sessions, commonly a small local database on the same machine. When a session begins or a relevant topic comes up, the agent recalls the facts that bear on the task at hand and they are placed into its context, where the model reads them like any other text. The frozen model still has no memory of its own. What changed is that the notebook was opened to the right page and set on the desk. The model appears to remember, but the memory lives in the store and the store handed it back.

The recall step has to be selective for the same reason the store has to be. You do not want every saved fact poured onto the sheet at once, because that would flood the context with irrelevance and crowd out the task. You want the few facts that matter to this moment. Good memory is therefore two disciplines at once: choosing well what to save, and choosing well what to bring back.

There is a quiet requirement hiding in the reading step: the fact has to still point at something real. A note that says “the tricky logic lives in the billing file” is worthless if the billing file has since been renamed and the note now points at nothing. This is why the best durable facts are anchored to the code they concern in a way that survives ordinary change. When a fact is tied to a specific piece of the system rather than to a file name that might move, it keeps its meaning as the code is reshaped around it, and a fact that keeps its meaning is a fact worth recalling months later. A memory that goes stale the first time someone tidies the code is barely a memory at all.

How Koragraph handles memory

Koragraph gives a coding agent durable memory in exactly this shape. Koragraph MCP is a local server that stores facts in a local SQLite database at ~/.koragraph/practice.db, and those facts persist across sessions and follow files even when they are renamed. Because it runs entirely on the developer’s machine with no cloud dependency, the memory stays local: no source code leaves the machine.

Of the tools Koragraph exposes to an agent, two are the memory tools: recall and remember. remember is the writer, the one action that saves a new fact into the store, and it is the only tool that writes memory. recall is the reader, the action that brings saved facts back into the agent’s context when they are relevant. Between them they cover the two halves this page described: write a lesson down once with remember, and pull it back when it matters with recall.

The split is deliberate. Keeping a single writer means there is one clear way a fact enters the memory, rather than many scattered paths that would be hard to reason about. And because the store sits beside the rest of what Koragraph knows about a codebase, a remembered lesson can be anchored to the actual code it concerns, which is what keeps it useful as the code moves and gets renamed around it.

Memory as a layer on top of understanding

It is worth being clear about what memory is and is not. Memory does not make the model less frozen or less stateless. The weights are still fixed and the model still forgets everything the instant a session ends. Memory is a layer built around that unchanging model: an outside store that holds the few hard-won facts worth keeping, plus the two actions that write to it and read from it. The model stays the same. Its surroundings get a notebook.

That layer is what turns a series of disconnected sessions into something that accumulates. Each session that ends by remembering one true, durable lesson leaves the next session a little wiser, not because the model learned, but because the store grew. Over time an agent working this way stops rediscovering the same fixes and stops walking into the same hazards, because the record of them is waiting to be recalled.

Two threads run out of this idea. The reason the model is frozen and forgetful in the first place, and what training and inference actually are, is the subject of training and inference. And the specific choice to keep an agent’s memory in a small local database on the developer’s own machine, rather than a distant server, is the subject of SQLite as the store, where the properties of that kind of store, local, durable, and quick, are exactly what durable memory needs.