I watched a junior developer ask our AI coding agent about the auth flow last week. The agent confidently suggested refactoring to use a centralized auth service.
We already have one. Built it two years ago. The agent had even helped maintain it in previous sessions.
But today? Today it had no idea that code existed. Every conversation starts from zero. Every session, the agent wakes up with amnesia, blinking at the codebase like it’s never seen it before.
This isn’t a bug. It’s not a context window thing we can engineer around. It’s baked into how these things work. And it explains why AI coding agents—despite being genuinely impressive—still can’t replace the developer who’s been on your team for five years.
The Developer Who Just Knows
You’ve worked with someone like this. Maybe you are someone like this.
They’re the person you ping when you’re stuck. Not because they’re smarter than everyone else, but because they just… know things. “Oh, we tried that approach in 2022—check the PaymentProcessor class, there’s a comment explaining why we went a different direction.” Or: “That utility already exists. It’s in the shared helpers folder. Sarah wrote it for the mobile team.”
No searching. No grepping through the codebase. They just know.
I used to think this was raw intelligence or years of experience. It’s not. I’ve seen brilliant developers who’ve been on a project for years but still can’t navigate it intuitively. And I’ve seen people with less “talent” who somehow became the human encyclopedia for their codebase.
The difference is memory. Not facts you consciously recall, but patterns that got built into your brain through repeated exposure. A spatial sense of where things live, how they connect, why they exist. It influences every decision, every code review, every architectural discussion.
AI coding agents don’t have this. They can’t have this. Not yet.
The Context Window Illusion
“But context windows are getting huge,” you might say. “Claude can handle 200k tokens. Surely we can just… fit more codebase in there?”
Let’s do some math.
A typical line of code is about 30-50 tokens. A 200k context window sounds massive until you realize that’s roughly 4,000-6,000 lines. A medium-sized codebase has hundreds of thousands of lines. A large one has millions.
You can fit maybe 1-2% of a real codebase in context. Often less.
And that’s assuming you know which 1-2% to include. The agent can’t “just know” that the auth utility exists if it’s not in context. It has to search for it. Which means it has to know to search for it. Which requires the exact knowledge it doesn’t have.
RAG helps—you can embed your codebase, build a vector database, let the agent search for relevant code. But this is reactive. The agent has to form a query. It has to know what it’s looking for.
It can’t stumble upon relevant context the way a human does when their brain goes “wait, this reminds me of…” You can only find what you think to look for.
The Session Boundary Problem
Even if we solved retrieval perfectly, there’s another issue: agents don’t learn.
When I work with a codebase for a week, I come out knowing more than when I started. Not just facts I could write down, but intuitions. The codebase “makes more sense.” I’ve updated my mental model.
When an AI agent works with a codebase for a week? It doesn’t accumulate anything. Each session starts fresh. The brilliant insight it had on Tuesday is gone by Wednesday. The context it built up in a long conversation evaporates when the session ends.
It’s like having a developer with severe amnesia. They might be brilliant in the moment—capable of impressive reasoning—but they can never build the accumulated understanding that makes senior developers so valuable.
Every Monday, they show up having forgotten everything from last week. Every conversation, you re-explain the architecture. Every task, they rediscover the patterns and conventions.
You can work around this. Extensive documentation. Carefully crafted system prompts. Memory systems that store and retrieve past conversations. These help. But they’re prosthetics for a missing capability, not replacements.
What’s Actually in Those Neural Weights?
Here’s the thing about current models: their weights are frozen at training time.
All the knowledge, all the patterns—they were baked in months or years ago using a massive dataset of code from across the internet. The model knows a lot about programming in general. It’s seen millions of codebases.
But it doesn’t know your codebase. Can’t know it, not in the way that matters, because it can’t update its weights based on working with you.
When a senior developer learns your codebase, their neurons literally change. New connections form. Existing ones strengthen or weaken. The experience becomes part of who they are as a developer.
When an AI agent “learns” your codebase, it’s just holding information temporarily in working memory. Nothing persists. Nothing updates. The model itself stays exactly the same.
This is why the same agent can help you refactor your auth system on Monday, and then suggest building one from scratch on Tuesday. It didn’t learn that you have an auth system. It just had that info in context temporarily.
The Tribal Knowledge Problem
Some of the most important knowledge in any codebase isn’t in the code at all.
Why did we choose PostgreSQL over MongoDB? Because three years ago, we tried MongoDB and hit scaling issues that took two weeks to diagnose. The developer who lived through that remembers. The codebase doesn’t.
Why is there a 500ms delay in the payment flow? Because we discovered that calling the payment provider too quickly after account creation sometimes fails due to propagation delays on their end. No comment explains this. Someone just… knows.
This tribal knowledge—the “why” behind decisions—is often more valuable than the code itself. It prevents teams from repeating mistakes, from reinventing solutions, from breaking things that look safe but aren’t.
AI agents have no access to this unless you explicitly tell them. And you can’t tell them what you don’t know they need to know. The senior developer’s value isn’t just in what they remember. It’s in knowing when something is relevant without being asked.
The Path Forward
I’m not saying AI agents are useless. They’re incredibly powerful. But there’s a ceiling to the current architecture, and we should be honest about where it is.
The real breakthrough will come when models can genuinely learn and retain knowledge about specific codebases over time. Not cached context. Not vector databases. Actual persistent learning that changes how the model thinks about your code.
Some work is happening here. Fine-tuning on proprietary codebases. Continued pretraining. Memory-augmented architectures. But we’re not there yet. Not close.
Until then, AI coding agents will be brilliant consultants who can help with any problem you put in front of them—but who show up to every meeting having forgotten everything from the last one.
Living With Amnesia
So what do we do in the meantime?
First, understand the limitation. Stop expecting your AI tools to “know” things they can’t possibly know. Provide context generously. Don’t assume the agent remembers anything from previous sessions.
Second, invest in the humans who do build this knowledge. The senior developers who’ve been around, who understand the why behind the code, who can make connections the AI never will—they’re more valuable than ever.
Third, write things down. If tribal knowledge matters (it does), and AI agents need explicit context (they do), then documentation isn’t just nice to have. It’s the bridge between human memory and AI capability.
The AI doesn’t remember why you chose PostgreSQL. But if someone wrote it down, the AI can at least read it.
Finally, be patient. The limitations are real, but they’re not permanent. The field is moving fast. Memory in weights will come eventually. When it does, AI agents will be fundamentally more capable.
Until then, we’re working with tools that have incredible capabilities and a peculiar form of amnesia. Use them for what they’re good at. Don’t expect them to be something they can’t yet be.
And maybe be a little more grateful for the senior developer who just… knows.