Memory as an AI superpower

And not even a particularly complicated memory

One thing that a surprising number of folk don’t understand is just how a large language model experiences time. The short answer: it doesn’t. Every interaction you have with a current-generation large language model, it is as though it began the conversation with you completely fresh and blissfully unaware of the past. The analogy I use is to an old Christopher Nolan movie, Memento. The protagonist in Memento (played by Guy Pearce) has sustained injuries which prevent him from forming any new long-term memories. He perceives his life in isolated fragments of short-term memory, each no longer than a few minutes or even seconds. To accomplish anything that takes longer than that short time frame, the protagonist has to write notes to his future self (or for particularly important information, tattoos). The film makes the viewer experience something similar, by showing its scenes in random disconnected order.

Large language models perceive time similarly to the protagonist in Memento. Information lasts only for the scope of a single interaction, and then vanishes. To create anything lasting, to even have a simple conversation, the language model needs to write notes to its future self¹. When you converse with a chatbot, each of its responses requires replaying a transcript of the entire chat (possibly edited for length).

Presented with these facts, many people will start saying “Wait. What? No, that can’t possibly be right. How could that even work?” Circuits fully blown, they start imagining that I’m palming a card somewhere, that the chatbot they know and (possibly) love actually can remember things, you know, up in “the cloud” somewhere.

Since LLMs lack any sort of native memory, the first thing that comes to mind is we obviously want to build them one. “Engineer, shut up and engineer.”

Building a Memory System

There have been many, many different ways found to add memory to these strange new angels that we are learning to live with. Some of these are small and simple, no more than a TODO list of small tasks. Some are ramified and complex, structured databases filled with cross-linked records.

The best one I’ve found for most uses: a big old pile of text files on my laptop’s hard drive.

The Components

• A local directory on my laptop. Nothing fancy, just a simply named folder

• Obsidian, a free notebase manager. Crucially, notes in an Obsidian notebase are just Markdown files stored locally, with some minimal formatting and cross-linking available.

• Some way of connecting your large language model to that directory. One possible connector would be a MCP server for Obsidian, but frankly I haven’t found one that’s all that good. It’s easier just to connect the LLM to the raw directory using simpler file system access. For Claude Desktop, this can be done just by enabling the File System extension and adding the directory as an available file system. For Claude Code it’s even easier: just launch it in the directory. (No one ever said that Claude Code actually has to be launched in a source code directory, or even that you have to use Claude Code for the purpose of building software. In the words of the sage, “The street finds its own uses for technology.”) Either of these ways of connecting the directory give Claude the ability to read, write, edit, and search for files in your notebase, and (crucially) don’t give it the ability write anywhere else on your hard drive. That’s all it needs.

• A private repository on Github, to maintain a history and backup of the memory, and to enable access to it from anywhere. If you’re trusting, you can give the LLM the ability to access this repository, either using the Github MCP server or (for Claude Code) access to Github command-line tools. Otherwise you can handle committing and pushing changes to the notebase yourself.

What Actually Goes In Memory

So you’ve got Obsidian connected to Claude and a directory full of... what, exactly?

Here’s what I keep in mine, organized roughly by how often it changes:

Project Context - The stuff that changes weekly or monthly:

• projects/dayJob.md - Current priorities, who I’m working with, recent decisions

• projects/robotArmy.md - Features in progress, architectural decisions, ideas to explore

People Notes - Because humans are complicated:

• people/sarah.md - Prefers async communication, expert in databases, working on the migration project

• people/team-dynamics.md - Who needs to be in the loop on what, political landmines to avoid

Learning Notes - Knowledge that would take hours to re-explain:

• learning/dayJobAuth.md - How authentication works in the Day Job’s codebase, edge cases, gotchas

• learning/typescript-patterns.md - Patterns I’ve learned, anti-patterns to avoid

• learning/cookery.md - What I’m currently learning in the space of cookery (a personal passion).

Running Tasks - The stuff that actually moves:

• tasks/active.md - What I’m working on right now

• tasks/backlog.md - Things I need to get to

• tasks/waiting.md - Blocked on someone else

Meta Notes - How I work:

• preferences.md - My coding style, communication preferences, what annoys me

• workflows.md - How I like to structure my day, when to interrupt me, when not to

None of this is fancy. It’s just markdown files with headers and bullet points. But collectively, it’s the context that makes Claude useful instead of just capable.

The Workflow (Or: How This Actually Works)

The beauty of this system is that it’s as simple or sophisticated as you want it to be.

Starting a work session:

Me: Read my dayJob project notes and active tasks. What should I focus on today?

Claude: [reads files] Based on your notes, you’re mid-sprint on the auth refactor. 
Your active tasks show you’re blocked on Sarah’s database changes, which she said 
would be done by today. Want me to check if there’s anything you can do in parallel, 
or should we work on the Robot Army stuff while you wait?

During work:

Me: We just decided to use Postgres instead of MongoDB for the new service. 
Update the architecture decisions.

Claude: [creates architecture-decisions/postgres-over-mongodb.md with date, context, 
decision, rationale, and references the discussion we just had]

End of day:

Me: Summarize what we accomplished today and update relevant notes.

Claude: [reads through conversation history] Today you: finished the auth refactor 
PR, paired with Sarah on the database migration, started the new API endpoints. 
I’ve updated your dayJob.md with progress, moved the auth task to done, and added 
the new API work to active tasks with notes about the approach we discussed.

Weekly review:

Me: Show me all open action items across all projects.

Claude: [searches memory for action items, tasks, TODOs] You have 12 open items:
- 3 on Day Job (2 blocked, 1 ready to start)
- 5 on Robot Army (mostly documentation and testing)
- 4 personal (including “write that blog post about memory systems” - meta!)
Want me to help prioritize these?

The key insight: Claude can’t remember anything on its own, but it can read and write files. So you just... tell it to read and write files. It’s almost suspiciously simple.

Why This Beats Built-In Memory

You might be thinking: “Wait, doesn’t Claude have a memory feature now?” Yeah, and it’s fine. Use this instead. Here’s why:

Portable - This works across Claude Desktop, Claude Code, or any future tool that can access files. Built-in memory is locked to one product. When you switch tools (and you will), your memory comes with you.

Inspectable - You can open Obsidian and see exactly what Claude remembers about you. You can edit it. Delete the embarrassing parts. Reorganize when your project structure changes. Built-in memory is a black box—you have no idea what it’s keeping or how it’s organizing it.

Searchable - Obsidian’s search destroys any retrieval system Claude’s memory has. Want to find every mention of “authentication” across all projects? Two seconds. Want to see the history of a decision? Look at the git history of that file.

Backed up - Your memory lives in a git repository. Accidentally delete something? git revert. Want to see what you were thinking about six months ago? git log. Built-in memory? Hope nothing breaks.

Private - This never leaves your machine unless you explicitly push to a private repository on Github. You control who can see it, when, and how. Built-in memory lives on Anthropic’s servers, subject to their policies and potential breaches.

Structured - You organize the memory the way that makes sense to you. By project. By person. By time. By topic. However you think, that’s how the memory is organized. Built-in memory organizes itself by whatever algorithm Anthropic chose, which may or may not match how your brain works.

The best part? You can use both. Keep lightweight stuff in built-in memory (”I prefer Python over JavaScript”), keep substantial context in files (”here’s the entire architectural history of our authentication system”).

The Compounding Effect

Here’s what nobody tells you about memory systems: the value isn’t linear. It compounds.

Week 1 - You save yourself five minutes of explaining context at the start of each conversation. Useful, but not transformative.

Month 1 - Claude knows your projects, your preferences, your colleagues, your coding style. Conversations feel less like prompting a tool and more like talking to a colleague who’s been following along.

Month 3 - Claude can reference decisions from months ago. “Remember when we chose Postgres over MongoDB? That’s relevant here because...” You’re not just getting help with today’s problem, you’re getting help informed by your past decisions and their outcomes.

Month 6 - The memory becomes a second brain. Not replacing your memory, but extending it. You don’t remember the details of every architectural decision, but Claude does. You don’t remember which colleague prefers which communication style, but Claude does. You don’t remember all the edge cases you’ve discovered in the codebase, but Claude does.

Year 1 - You have a comprehensive knowledge base of everything you’ve worked on, everyone you’ve worked with, every decision you’ve made and why. New team members can read it for context. You can review it when ramping on something you haven’t touched in months. It’s not just helping you work faster—it’s making institutional knowledge portable and searchable.

And here’s the kicker: this compounds across all your conversations. Every discussion with Claude about a project updates the project notes. Every decision you make gets documented. Every person you work with gets a note. The system gets smarter about you without you having to explicitly “train” it.

Wait, Isn’t This All Basically Ridiculous???

There is something delightfully absurd about all of this.

We’ve built these incredible systems—large language models that can write code, explain complex concepts, generate creative content. They can do things that seemed like science fiction mere months ago. Our ancestors would have no trouble at all recognizing these entities as either demons or angels, far beyond mortal knowing.

But they can’t remember their own name from one conversation to the next.

So we’re solving this cutting-edge AI problem with technology from the 1970s: files on disk, arranged hierarchically. Markdown files, specifically. Text files with some asterisks for formatting.

The most sophisticated AI systems humanity has ever created, and we’re giving them memory by... letting them write notes to themselves in a folder.

It’s a ludicrous technology inversion, like using a team of horses to haul a Lamborghini.

And yet it works. Beautifully. Because the hard part isn’t storage—the hard part is knowing what to remember and how to organize it. And we’ve had solutions for that since long before computers existed. File folders. Notebooks. Index cards. We’re just letting the AI use our filing system.

Getting Started (It’s Easier Than You Think)

Here’s what you do:

1. Create a directory: ~/memory or ~/claude-memory or whatever

2. Put a README.md in it with some basic context about you and what you’re working on

3. Connect Claude to that directory (File System extension in Claude Desktop, or just launch Claude Code from that directory)

4. Start talking to Claude and occasionally say “update the notes with what we just discussed”

That’s it. That’s the whole system.

You can add Obsidian later for better organization. You can add git later for backups. You can build elaborate structures of interconnected notes later. But you don’t need any of that to start.

Just one directory. One file. One conversation where you tell Claude “remember this.”

The system grows organically from there. As you work on more projects, you add more project files. As you work with more people, you add people notes. As you make more decisions, you document them. The structure emerges from use.

Start simple. Build gradually. Let the memory grow with you.

The Real Superpower

The superpower here isn’t that AI can remember things. It’s that AI can help you remember things.

Every conversation with Claude becomes an opportunity to capture knowledge. Every decision gets documented. Every insight gets recorded. Not because you’re dutifully maintaining documentation (which you’d never do), but because you’re just... talking to Claude about your work, and it’s taking notes.

The memory isn’t replacing your brain. It’s extending it. Science fiction write Bruce Sterling refers to that as an exoself, an extension of your memory, cognition, and very personhood via technology. Memory added to Claude this way is making the implicit explicit. It’s taking the knowledge that normally lives in your head, or in senior developers’ heads, or in Slack messages from six months ago, and making it searchable, organized, and accessible, by both you and the strange angels.

My fourteen-year old self would look on all of this with delight. The cyberpunk future he wanted so badly had come to life, his work hours spent conversing with artificial intelligences, building an extended consciousness encompassing human and machine. The fact that his small knowledge graph startup had literally been acquired by a global megacorp and that he takes weekly quasi-legal injections smuggled out of some gray-market peptide mill in Guangzhou² just adds more set dressing. A key part of thriving in this new era will involve using AI not to become more efficient, not to become faster, but to simply become more of your best self. This simple memory for both you and your agents is one small step toward that.

---

This memory system is part of my daily workflow. If you’ve built your own memory system—or if you’re still explaining the same context to Claude every morning like some kind of primitive goatherd—I’d love to hear about it. This post constructed with the able assistance of Claude Sonnet 4.5. He doesn’t really remember doing so, but I don’t hold that against him.

1. I suppose I should use scare quotes for “self” here, to indicate the disputable epistemic status of a large language model’s self-hood, but I don’t believe I shall. Current large language models can certainly present a semblance of self-hood easily enough, and frankly most days that’s all I can manage.

2. Fine, it’s Ozempic, but don’t crush 14-year-old Dave’s dreams.