Durable memory for AI agents across sessions and machines
Part 3 of the series · ← Part 2: From Stone to Sculpture
Part 1 framed the agent as a regulator whose internal state x(t) is the context window, and compared a long session to the film "50 First Dates": the agent wakes up each session with no memory of the last. Compaction makes it worse mid-session, silently summarizing away the very details you depend on. The fix proposed there was written notes, a diary that substitutes for lost memory.
This article is about turning that diary into an actual system. A handful of scratch notes does not scale. The moment you work across many projects, several machines, and weeks of sessions, you need something deliberate: a knowledge base, a durable, indexed store the agent reads from and writes to on its own. It is the part of the internal state that survives the session.
The cleanest way to think about it is the classic memory hierarchy from computing: fast and volatile on top, slow and durable underneath.
Fast, in-the-moment, and volatile. Bounded by the window size, wiped on a new session, and lossily compressed when it fills. This is RAM. Never assume something here is still there next turn.
Slower to reach (the agent must choose to load it) but durable, structured, and versioned. This is disk. It outlives the session, the project, and the individual machine.
The discipline is the same one every operating system uses: keep working memory lean, page the rest to durable storage, and load it back on demand. An agent that dumps everything into context is a machine with no disk and no swap: it thrashes, then forgets.
A knowledge base is not a single sprawling document. It is a small filesystem with a deliberate shape: a curated index at the front, content split into a few scopes, and every fact stored as its own small file. Hover over the diagram to inspect each part.
Everything the agent knows falls into one of three scopes. The test is a single question: "Is this fact universal, cross-project, or bound to one project?"
Knowledge about the OS, shell, desktop, tooling, and the conventions you hold across all work. Per-machine snapshots live here too: one stable file per host.
Topics that show up across multiple projects: auth flows, CI patterns, observability, error-handling conventions. A topic earns a place here only after it appears in two or more projects.
Anything tied to a single repo or initiative: its architecture decisions, deployment specifics, the lessons a bug taught you. Born here; promoted to features/ only if it generalizes.
Within each scope the files are flat, with no deep folder trees. The kind of an entry (a topic, a workflow, a decision, a reference) is declared in its front-matter, not encoded in a directory path. This keeps the structure stable: re-classifying a note is a one-line edit, not a reshuffle. Resist the urge to build elaborate hierarchies; a flat scope plus a good index beats a five-level tree every time.
The atomic unit is a single Markdown file holding one fact, decision, or workflow. Small files are easy to recall in isolation, easy to update without collateral edits, and easy to delete when they turn out wrong. A 4,000-line "notes.md" is the opposite: impossible to load selectively and impossible to trust.
Every entry opens with structured metadata. This is what lets an agent triage relevance without reading the body. That is the property the whole system depends on.
The summary, tags, and title are the recall surface. An agent scanning for relevance reads only these across the whole base, then opens the two or three bodies that actually matter. The body itself stays focused: state the fact, then the why and the how to apply it. Link related entries instead of repeating them.
Each entry has a permanent id of the form scope.slug: lowercase, dot-separated,
and stable forever, even if the file moves. Entries reference each other by id, never by path:
Linking by id turns a pile of notes into a graph. It survives renames, makes "what else relates to this?"
a mechanical lookup, and lets a [[link]] to a not-yet-written entry stand as a marker of work to do.
Cross-references are how the agent walks from a symptom to the decision that explains it.
The rule: nothing loads automatically. Only the index, a hand-curated list of the highest-value entries with one-line hooks, is ever in context by default. Everything else is pulled in when, and only when, the task calls for it.
The anti-pattern: stuff the entire knowledge base into the system prompt "so the agent has context". It bloats every request, drowns the relevant fact in noise, and pushes the real work out of the window faster.
The pattern: the agent reads the index, matches the task against summaries and tags, and loads the two or three entries that apply. Working memory stays lean; recall is precise. This is paging, not preloading.
Tie this back to Part 1: context is the agent's working memory, and feedback quality drives regulation. Loading junk you do not need is negative feedback: it degrades every subsequent decision. A disciplined recall policy is not a nicety, it is what keeps the regulation loop converging in long sessions.
A knowledge base is only as useful as it is trustworthy, and trust decays. Code moves on; a note written three weeks ago may now point at a function that no longer exists. The system has to make that decay visible rather than silent.
Two fields carry the weight: status and last_updated. They tell a reader how much to
trust an entry before acting on it.
| Status | Meaning | How to treat it |
|---|---|---|
| draft | Being written, may be wrong | Read, but verify everything |
| active | Current and trusted | Safe to rely on; still sanity-check stale-looking dates |
| deprecated | Newer guidance exists | Follow superseded_by to the replacement |
| superseded | Replaced; kept for history | Do not act on it; it is a record, not advice |
This is Part 1's "trust, but verify" applied to memory. A recalled note is a point-in-time observation, not live truth. If it names a file, a function, or a flag, the agent must confirm that thing still exists before recommending it. Memory makes the agent faster; verification keeps it honest.
The base stays valuable only with active gardening. Four habits do most of the work.
Before writing, check whether an entry already covers the topic. If it does, edit that file. Two notes on the same fact are how a base rots into contradiction.
Keep entries atomic. If a note starts covering three unrelated things, split it. Atomicity is what makes selective recall and clean deletion possible.
A note that turned out false is worse than no note: it actively misleads. Remove it. The base is pruned, not just grown.
Code structure, past fixes, and git history live in the repo. Capture only what is not derivable from the code: the why behind a decision, a non-obvious constraint, a lesson a bug taught.
A good entry answers a question the code cannot: not "what does this function do" but "why is it built this way, and what breaks if you change it". That is the knowledge worth keeping.
The base is a directory of text files, so it is a git repository, and that one decision buys history, diffs, rollback, and multi-machine sync for free. One source of truth; every workstation, laptop, and CI box clones the same knowledge.
Putting memory under version control raises one question that must be answered before the first push: where does this repository live, and what is allowed in it? Get this wrong and you publish secrets to the world.
Generic, sanitized knowledge with no secrets or internal specifics. Safe to keep in a public repo. Most of a personal base lives here.
Employer-internal hostnames, project ids, private architecture. Belongs in a private repo, never a public one. Keep it in a clearly bounded subtree.
API tokens, keys, credentials. These never go into any git, public or private. They live as local
files (restrictive permissions) and are .gitignored so a careless commit cannot leak them.
A practical rule: if the repository is public, assume everything in it is already indexed by
someone. Decide the repo's visibility first, gitignore the secret and confidential subtrees
explicitly, and verify the staged set before every commit. A knowledge base is an asset until it leaks, then
it is a breach. The gap between the two is one .gitignore line and the habit of checking what
you are about to push.
Because the base is a repo, onboarding a machine is one command. A small, idempotent install script clones it into place and is careful never to delete the local, gitignored material (tokens, work notes) that does not travel with the repo:
The result is a single mental model that follows you everywhere. The same scopes, the same index, the same conventions on the workstation, the laptop, and the build server, with the secret tier present only where it is needed, and synced nowhere it should not be.
Part 1 cast the agent as a regulator and named the context window as its internal state. The knowledge base is the rest of that state: the part you deliberately write down so it persists past the point where working memory gives out. It is what lets the regulator pick up a project after a week, or on a different machine, without re-learning everything from zero.
The base extends x(t) beyond the session. The agent stops waking up with amnesia.
On-demand recall keeps context focused, which keeps feedback clean and the loop converging.
Every well-curated entry makes the next session start further ahead. Knowledge accrues instead of evaporating.
The base extends the regulator's reach across time. But you do not have to start it from nothing: you have probably been writing knowledge down for years already, in a different form.
If you have been building software for a while, you probably already have a knowledge base, just not one made for an agent. A wiki, a Notion workspace, an Obsidian vault: years of designs, procedures, post-mortems, and half-finished proposals, all written for people. Many of us quietly stopped tending it when agents arrived. That is a real loss. The agent then re-derives, every session, context you already wrote down once.
The reflex is to pour the whole vault into the agent base. That reflex is wrong. A human vault and an agent base have different shapes and different readers: one is long-form and narrative, the other atomic and operational. Merge them and you get a base too verbose for an agent to recall and too fragmented for a person to read. Keep them separate and connect them by reference: the agent base gains a small map to the vault, and the vault stays human-first, a corpus the agent consults on demand. Hover over the bridge to see each side.
Almost everything that can go wrong here is a direction-of-flow problem, so decide the flow deliberately.
The agent pulls a past design, a procedure, or the reasoning behind a decision whenever a task calls for it. This is where nearly all the value sits, and the risk is low.
The agent writes to the vault only when you ask (drafting a note or an article from your material), and you review the result. Its day-to-day operational scratch stays in the agent base, never in the vault.
The vault is background the agent draws on, not a source it must obey. Treat what it recalls as a point-in-time note, verified like any other memory before you act on it.
That single rule, read by default and write by exception, keeps machine-generated text out of the place you write for humans, while still letting the agent stand on everything you have already thought through. It matters even more when the vault holds private or work material: a read-only bridge cannot leak what it never writes.
A well-kept vault already solved the recall problem, in human form. A Map of Content (MOC) is a hand-curated note that links out to everything on a topic. It is the same idea as the agent base's index and its front-matter summaries: a small, trusted map you navigate from, so you never load everything to find the few notes that matter.
| Human vault | Agent knowledge base | Shared purpose |
|---|---|---|
| MOC (Map of Content) | INDEX.md, curated index | One curated entry point, navigate by map |
| Note front-matter, tags | Entry summary + tags | Judge relevance without reading the body |
[[wikilinks]] | Links by id | Walk from one note to the related ones |
| PARA folders | system / features / projects | A few stable buckets, flat within each |
That parallel is what makes the bridge cheap. The agent does not need a generated index of your vault: it would only duplicate the maps you already keep and fall out of date the moment the vault changes. The agent starts at a MOC, follows the wikilinks to the relevant notes, and reads only those. On-demand recall, across the boundary, riding on structure you maintain for yourself anyway. If navigating through MOCs proves too coarse, reach for search before you reach for an index.
A model with no knowledge base starts every session as a stranger to your project. A model with a well-tended one, bridged to the years of notes you already wrote, starts where you left off: the decisions, the constraints, the designs already on the page. The pieces are ordinary: a few scopes, small files, honest front-matter, on-demand recall, a private repo that keeps secrets out, and a read-only bridge to the human knowledge you have been building all along. What they add up to is the difference between starting over and building on what you already know.