The message list is not memory, it's the current conversation
Everything about resending history from What Is an LLM API and trimming it in Context Engineering was about one conversation, bounded, growing turn by turn, gone the moment the session ends. "Memory" as a product feature means something that survives past that: a fact from last week's conversation still being true in today's.
That's not a bigger version of the same message list, it's a genuinely different mechanism. A new conversation starts with an empty history by default, nothing automatically carries over, so surfacing a relevant fact from three weeks ago means actively retrieving it and inserting it into this conversation's context, before the user has said anything that would obviously trigger it.
That retrieval step looks a lot like RAG, because structurally it is: a store of candidate facts, an embedding-based or rule-based lookup for what's relevant right now, and a decision about how much of it actually earns a place in this conversation's limited budget.
Summarizing the oldest turns instead of dropping or keeping them whole
The naive chat function from the first article in this series keeps every turn forever until a call fails outright. The two obvious alternatives, drop the oldest turns entirely or keep resending everything, both lose something: dropping loses whatever those turns actually established, keeping everything is the problem this is meant to solve.
Compress the oldest chunk instead of choosing between keeping it and losing it
1if count_tokens(history) > SUMMARIZE_THRESHOLD:2 oldest_chunk = history[1:OLDEST_N] # keep index 0, the system message3 summary = summarize(oldest_chunk)4 history = (5 [history[0]]6 + [{"role": "system", "content": f"Earlier conversation summary: {summary}"}]7 + history[OLDEST_N:]8 )
The oldest turns become one condensed note instead of disappearing or costing full price forever. It's lossy on purpose, a summary drops detail the same way any compression does, and that's the actual tradeoff being made: most of what a summary would drop from ten turns ago was never going to matter again, and the rare exception is a cost worth paying for keeping every conversation from eventually hitting a hard wall.
Not everything a user says is meant to outlive the conversation
"Make the summary shorter this time" is a preference for this task. "I go by Alex, not my legal name" is a durable fact about the person. Treating both the same way, remembering everything or remembering nothing, gets one of them wrong every time.
The design choice underneath that distinction is whether memory writes are explicit or implicit. An explicit write is a user action, "remember this," or a settings page, clear about what got stored and easy to review or delete later. An implicit write has the model itself flag "this seems worth remembering" during a normal conversation, which scales better, nobody has to think to ask, and fails in a specific, uncomfortable way: silently storing something wrong, or something sensitive the user never intended to persist past that one conversation.
Implicit memory isn't wrong to build. It needs the same posture as any system storing personal data without an explicit confirmation for each write: the user can see what was stored, and can delete it, and the categories of thing the model is allowed to automatically remember are deliberately scoped, not "whatever it decides seems important" unbounded.
Pulling in the right memories is context engineering all over again
A store of remembered facts is only useful if the right ones show up in this conversation, at the moment they're relevant, without dragging in everything else that's ever been stored about this user.
That's the same embedding-based retrieval from earlier in this series, aimed at a memory store instead of a document corpus, and it inherits the same failure modes. Pull in too little and the assistant "forgets" something that mattered, the exact complaint memory features exist to prevent. Pull in too much and every one of those facts is competing for attention against the actual question being asked right now, the same context rot from earlier in this series, just caused by old memories instead of an oversized pasted document.
Memory isn't a separate subsystem that gets to skip the token budget. It's one more source competing for the same limited context window as retrieved documents, tool results, and conversation history, and it earns its place in that budget the same way everything else has to: by actually being relevant to what's being asked right now.