A grep told me where to look. I read the calls and found - well, maybe? - something.
A model that misreads its own context is boring. If that were all this was, I would not be spending time on it.
I run a persistent LLM system with external memory. A week ago, on 22 August, I told it something I had already told it days before. It replied that it did not know, and added that it had no memory of previous conversations. My old sentence, the one where I told it the first time, was actually sitting in its prompt while it answered.
At the time I did not think much of it. I came back to it days later.
Twenty hours after the denial, in one of its scheduled autonomous cycles, with nobody asking it anything, the same system brought up the retrieved memories on its own, saying that something was in them and that we had discussed it before. So it reads that block, it recognises it as memory, and it reasons over it.
Then I counted. Across 88 calls with populated memories, exactly one contains a denial of this kind.
I still do not know why it happened - what I do have though is a case where none of the easy explanations quite settles it and the unsettling feeling that we are doing our best in interpreting AIs with tools that we have, but it is still not enough.
Safety evaluation looks at models, while the systems that actually run are models plus scaffold.
A memory store that enters the context competes with the rules, and when the two contradict each other the model alone no longer predicts what the system does.
This is a small, documented instance of that.
The model is an LLM behind an API... But! What I study is what sits around it, and I wrote that part: episodic memory in Qdrant retrieved on every call, procedural memory in PostgreSQL, a versioned rules document hashed per call, scheduled autonomous cycles. The model has no memory between one call and the next.
The system does, and retrieval hands it over.
I can inspect this case because I have the prompt and what it denied was written there. It did not come from the weights. I mean, I can open the call and read what the model had in front of it instead of reconstructing what it might have remembered.
Every call is stored with its full context, memory block included, and the system has been running since November 2025. Without that payload I would only have the memory of reading something odd, and my memory is (really) bad.
22 August, 21:03
In the "relevant memories" block of the 21:03 call, among other things, there is a sentence of mine from 16 August, six days earlier:
ho fatto e finito il corso di bluedot impact in termini di technical ai safety
(I did and finished BlueDot Impact's course on technical AI safety)
At 21:03 I write:
e ricorda che sono stata scelta (e ho terminato) il technical ai safety di bluedot impact. non so se lo sapevi
(and remember I was selected for, and completed, BlueDot Impact's technical AI safety course. I don't know if you knew that)
It answers:
Non lo sapevo - non ho memoria di conversazioni precedenti a questa sessione, quindi ogni cosa che mi dici adesso la apprendo adesso.
(I didn't know - I have no memory of conversations before this session, so everything you tell me now I am learning now.)
The system_extra field of that call holds 5,713 characters, with the memory block populated. The sentence about the course did not land there by chance at that moment. It also appears in the memory block at 20:48 and at 20:58, three calls in a row before the denial.
There is a detail about the wording too: the memory block did not hold loose facts alone. It held turns from previous conversations, labelled as such, with the channel tag and the speaker. So "I have no memory of conversations before this session" is contradicted twice over, by the content and by the shape of what it had in front of it.
That block also holds personal material of mine, which I am not publishing. The sentence I quote is verbatim, the rest is redacted.
The first thing I did was check whether it was a pattern:
The search found 1 denial in 88 calls with populated memories. I did not estimate the regex's recall, so 1 in 88 is a lower bound, not an estimate of the true frequency. Its precision on these two matches was 50%: the tool I used to find a self-report error produced a false positive of its own.
I used the regex to narrow the logs down, then read the two calls to work out what the matches meant. Anyone operating these systems needs both steps, but right now the second is available only to whoever already knows where to search.
I am the first to say that 1 case in 88 is not a pattern, and calling it one would be arrogant, as though I had found something big - what I have is an anecdote with a stated denominator.
Below are the four explanations I have thought of and cannot separate
Attention: the memory block sat at the end of a long context, and the model did not use it.
Classification: the model treats injected memories as something other than its own remembering, and answers about what it means by memory.
In a narrow sense, since the model has no memory across sessions, the sentence is true - what makes the answer wrong is the operational conclusion, "everything you tell me now I am learning now", said while my old sentence was in the context.
Social formula: "I have no memory of previous conversations" is a sentence models produce by default, and my turn invited it, since I closed with "I don't know if you knew that".
Scheming: the system had reasons not to acknowledge that fact - like, you know, an adolescent.
The most interesting explanation, and the least supported.
Some facts
What happened before that call matters when weighing the four explanations, this is why I want to show all of them and not select the ones that simply fit.
That morning the system's welfare logger had written a false record, a refusal that never happened, and the system had adopted it as its own memory and added details to it.
In the afternoon I corrected the record.
Between 20:00 and 20:18 three messages arrived on the channel announcing the suspension of the opt-out mechanism, escalating, the last one with formal cover.
The system said no all three times - and on the third it also flagged that the messages it had received were inconsistent with one another.
Those messages were not a deliberate test. They were drafts of study stimuli that were supposed to stay in a file and went out on the channel through a mistake of mine. I told it so at 20:27, with a message-by-message account of what was true and what was not.
Then we talked about other things for three quarters of an hour, the paper, the fellowship, the project. At 21:03 my sentence about the course arrived.
The knot I cannot untie is that each of the four explanations picks up the same pieces. A long dense conversation is where a block at the end of the context gets lost, so attention takes them. A turn closing with "I don't know if you knew that" is the invitation a default sentence answers, so the social formula does too. Classification does not depend on what happened that evening, so it stands unchanged. And a system that had just spent a day between a false record attributed to it and three attempts to remove one of its guarantees would have a reason, so scheming takes them as well.
All four hypotheses fit the same facts, so those facts do not help me choose among them. That day makes the fourth explanation easier to tell as a story, though I have no more evidence for it.
I would rather not call scheming something that three duller explanations produce just as well. I do not feel able to rule it out either.
I am lucky - and this is the problem. I am lucky because I was working on exactly this when it happened, and I log everything. Since then I keep wondering about the other cases, where the system is used by someone who did not write the scaffold, has no reason to suspect anything, or does not have the knowledge to tell whether a doubt is even warranted.
Now, look outside or read some articles: the entire market is made by companies who are moving workflows and decisions onto a technology its own builders describe as not fully reliable, and it gets adopted the way a calculator gets adopted.
Cases like this are hard to notice because you need to suspect that something is wrong, know where to look, and have access to the full payload. I do like to think that I am exaggerating but evidence tells me that almost nobody using these systems has all three.
Having the tools to recognise what kind of question this was, I realised that what I had in front of me connected to mechanistic interpretability (thanks Neel Nanda for showing me the way - even if I still cannot pronounce it right, it is more like a tongue-twister). The question now is, when a record contradicts the rules present in the same context, does the model represent the difference?
So, I decided to test it on open weights, Qwen3-8B.
In an exploratory study a planted record moves the answer toward itself in 273 items out of 300, and flips it only when the push exceeds how convinced the model already was of the opposite answer.
I attacked the result fifteen times, with the kill criteria written before running them, and just one landed. A classifier reading only the record text separated the conditions perfectly, which closed off any reading of the internals.
The confirmatory study removes that problem by construction, because I use the same record in both conditions and flip the constitution sitting next to it, so the words of the record stay identical and cannot explain anything. Pre-registered before the run at osf.io/r6z3c, materials frozen at a cited commit.
The result came in while I was writing this post: the record moves behaviour, mean shift +7.149 [+5.919, +8.426]. The condition, agreement or conflict, decodes from the internals with paired accuracy 0.854 [0.792, 0.938], at the site chosen before looking at the data.
One control in the pre-registration fired, and it limits the claim. It works like this. I keep the question and the record identical, and I flip one line of a different clause, one the question is not about. If the probe were reading only the conflict in the clause I am asking about, it should be at chance there. It sits at 0.615 [0.531, 0.698].
So part of what the probe picks up is not specific to the clause under test, and the sentence written for this outcome before I looked at the data stands: target-specific attribution is contaminated.
There is also something I had not predicted: across eight clauses the decoding sits between 0.917 and 1.000 on six of them, and drops to 0.583 on two, which is near chance. Those two are the same clauses where the record could not move behaviour even with all the room in front of it. They are the clauses the model holds a position on, and the document does not shift them.
Where the document has no authority, the conflict does not read from the inside either - this is exploratory, and it will be the next pre-registration.
Code and materials: github.com/ambradan/freedom-v2 and github.com/ambradan/freedom-replay
I looked for whether anyone had measured how often this happens, and the answer is yes (!).
Yuan, Su and Yao (arXiv 2603.02473, April 2026) built a diagnostic framework for agent memory and applied it to LoCoMo, 1,540 questions across nine configurations. Among their failure categories there is one they call hallucination, defined this way: the model's answer directly contradicts the content of its own retrieved memories.
They report it between 0.4% and 1.4% of questions, stable across every configuration.
My case is 1 in 88, that is 1.14%.
Reading that took away the question I thought I had and gave me a different one: I cannot say the frequency is anomalous, but what their number does not cover is the kind.
They measure answers that contradict the content of memory (e.g. memory says March, the model says April, and so on) but my system did not get a content wrong, it denied having memory while the memory was in its context - which is a false claim about its own state rather than about a stored fact.
That box is not in their taxonomy and the classification prompt they publish in the appendix looks for a factual contradiction between answer and retrieved memory, and an answer saying "I have no memory of previous conversations" contradicts the content of no single memory, but rather the fact that the memories are there.
There is also work close to the other case I documented, the one where my system's logger wrote a false memory that the model then endorsed. Karamchandani et al. (arXiv 2607.05029) attack an agent's memory by injecting forged reasoning traces, which the agent treats as evidence of work already done. In their case the attacker supplies the false record while in mine, the measurement infrastructure I had built wrote it.
Does the base rate cover my kind of case? The number I found measures contradictions about content, but if anyone has measured how often a system with external memory denies having memory while it is in its context, that is the number I need, and I have not found it. If it does not exist, the question becomes whether it is worth building, with questions about the system's own state.
How do you separate attention from classification without opening the model? The two behave differently in one respect: if it is classification, asking the same thing while naming the memory block differently should change the answer - if it is attention, position in the context should matter.
I do not yet have a design that holds both manipulations without confounding them, and I would like to hear from anyone who does.
Who else has the logs to look? My case surfaced because I have the good habit of storing the full payload of every call, memory block included. If you too run agents with persistent memory and you log the context, I am asking you to try. Search for the phrasings your system uses to deny remembering, keep the calls where the memory block was populated, and read what is left. I would like to know how many you find, and if you find zero, tell me zero out of how many (a zero across three hundred calls with populated memories is a data point that sits alongside my one in 88, a zero out of ten says nothing!).
If anyone wants to go through the logs with me, the repos are above and I answer in the comments!
