Your Agent Forgets You the Second You Close the Tab

Open a chat with any agent you have built, tell it your name, your stack, and the three things you care about, and have a genuinely useful conversation. Then close the tab. Come back tomorrow and it greets you like a stranger. Same name, same stack, same three things, all gone. You explain yourself again from scratch, every single time.

That is not a bug in your prompt. It is the default behavior of the thing you built on. Large language models are stateless. Each API call gets a context window and returns a completion, and nothing survives to the next call. The reason a chat feels like it remembers anything is that you are quietly resending the entire conversation on every turn. The moment that history stops being sent, whether because the session ended or the window filled up, the memory was never real to begin with.

For a demo, nobody notices. For a product, it is the difference between a tool people rely on and a toy they try once. This is the agent memory problem, and it is one of the defining production challenges of 2026.

The business problem: an agent with no memory cannot build a relationship

Think about what memory actually buys you, in business terms, not research terms.

A support agent that remembers a customer has asked about the same billing issue twice this month does not make them start over. A coding assistant that remembers you switched from REST to GraphQL last week stops suggesting the old pattern. A sales agent that remembers a prospect's budget and timeline picks up the thread instead of re-qualifying them. Every one of these is retention, conversion, or hours saved, and every one of them is impossible if the agent's knowledge of the user resets to zero at the start of each session.

The naive fix is to make the context window do the job. Just keep the whole history and send it every time. This falls apart on all three axes that matter.

Cost. You pay per input token on every call. If you carry a growing transcript into every turn, your per-request cost climbs with the length of the relationship, which is exactly backwards from what you want. Your best, most engaged users become your most expensive.

Quality. More context is not better context. Bury the one fact that matters, that the user is on the enterprise plan, under nine thousand tokens of small talk from three sessions ago, and the model's odds of actually using it drop. This is the same failure that makes retrieval systems return the wrong chunk, and it gets worse as the window grows, not better.

It still does not persist. Even a million token window starts empty when the session restarts. A bigger window buys you a longer single conversation. It does not buy you memory. Those are different problems, and conflating them is how teams end up paying frontier prices for something that still forgets.

Real memory is not a bigger window. It is a separate store that holds a small number of durable facts, outside the model, that you retrieve from selectively and inject only when they are relevant to the current turn.

The approach: extract, store, retrieve, consolidate

A memory layer is four moving parts, and it helps to name them before writing any code.

Extract. After a conversation, or during it, pull out the durable facts worth keeping. Not the whole transcript, just the things that will still matter next week.

Store. Save those facts somewhere queryable. In practice that means embedding them and putting them in a vector store, so you can later find memories by meaning rather than exact words.

Retrieve. At the start of a turn, take the user's current message, find the handful of stored memories most relevant to it, and slot them into the prompt.

Consolidate. When a new fact contradicts or updates an old one, reconcile them so the store does not fill with stale, conflicting junk.

The taxonomy the field converged on over the last year sorts memories into three kinds. Episodic memory is specific events ("on July 3rd the user asked to cancel Pro"). Semantic memory is distilled facts ("the user prefers concise answers and works in healthcare"). Procedural memory is learned how-to knowledge, the steps that worked before. Most teams should start with semantic memory. Distilled facts are the cheapest to store, the easiest to retrieve, and the most reusable, and they get you most of the value before you touch the harder kinds.

Let us build the semantic version.

Step 1: extract durable facts from a conversation

The mistake here is storing raw messages. Raw messages are noisy, they are long, and half of any conversation is throat-clearing you never want to recall. Instead, run the exchange through the model and ask it for facts.

# pip install anthropic
from anthropic import Anthropic
import json

client = Anthropic()
MODEL = "claude-opus-4-8"

def extract_memories(user_id: str, conversation: list[dict]) -> list[str]:
    """Turn a chunk of conversation into a few durable, standalone facts."""
    transcript = "\n".join(f"{m['role']}: {m['content']}" for m in conversation)
    resp = client.messages.create(
        model=MODEL,
        max_tokens=500,
        system=(
            "Extract durable facts about the user worth remembering across "
            "future sessions: preferences, stable context, decisions, goals. "
            "Ignore small talk and anything true only for this one exchange. "
            "Each fact must stand alone without the surrounding conversation. "
            'Return a JSON array of strings. Return [] if nothing is worth keeping.'
        ),
        messages=[{"role": "user", "content": transcript}],
    )
    text = "".join(b.text for b in resp.content if b.type == "text")
    try:
        return [f.strip() for f in json.loads(text) if f.strip()]
    except json.JSONDecodeError:
        return []  # never let a bad parse poison the store

Two details matter. First, the "must stand alone" instruction. A memory that reads "they said yes to that" is useless six sessions later because "that" is gone. You want "the user approved migrating the billing service to Postgres." Second, the explicit permission to return nothing. Most casual turns produce zero durable facts, and an extractor that feels obligated to invent something will slowly fill your store with noise.

Step 2: store facts as embeddings

To retrieve by meaning, you embed each fact into a vector and keep it alongside the raw text. Any vector store works; here is the shape of it with a simple in-memory store and an embedding model, so the logic is visible rather than hidden behind a service.

# pip install voyageai numpy
import voyageai
import numpy as np

vo = voyageai.Client()  # Anthropic's recommended embeddings partner

def embed(texts: list[str]) -> np.ndarray:
    result = vo.embed(texts, model="voyage-3", input_type="document")
    return np.array(result.embeddings, dtype=np.float32)

# In production this is your vector DB. The interface is what counts:
# (user_id, fact_text, vector). Always scope retrieval by user_id.
MEMORY_STORE: dict[str, list[tuple[str, np.ndarray]]] = {}

def save_memories(user_id: str, facts: list[str]) -> None:
    if not facts:
        return
    vectors = embed(facts)
    bucket = MEMORY_STORE.setdefault(user_id, [])
    for fact, vec in zip(facts, vectors):
        bucket.append((fact, vec))

The one non-negotiable is scoping by user_id. Memory that leaks across users is not a quality bug, it is a privacy incident, and it is the kind that ends up in a postmortem with legal on the call. Every query into this store must be filtered to the current user before similarity ranking, not after.

Step 3: retrieve only what is relevant, then answer

Now the payoff. When a new message comes in, embed it, find the closest few memories for that user, and put just those into the system prompt. Not the whole store. The point of all this machinery is to inject three facts, not three hundred.

def recall(user_id: str, query: str, k: int = 3) -> list[str]:
    """Return the k most relevant stored facts for this user's query."""
    bucket = MEMORY_STORE.get(user_id, [])
    if not bucket:
        return []
    q = embed([query])[0]
    facts, vecs = zip(*bucket)
    matrix = np.vstack(vecs)
    # cosine similarity; voyage vectors are already normalized
    scores = matrix @ q
    top = np.argsort(scores)[-k:][::-1]
    return [facts[i] for i in top]

def answer(user_id: str, message: str) -> str:
    memories = recall(user_id, message)
    memory_block = (
        "What you know about this user from past sessions:\n"
        + "\n".join(f"- {m}" for m in memories)
        if memories else "No stored memories about this user yet."
    )
    resp = client.messages.create(
        model=MODEL,
        max_tokens=1000,
        system=f"You are a helpful assistant.\n\n{memory_block}",
        messages=[{"role": "user", "content": message}],
    )
    return "".join(b.text for b in resp.content if b.type == "text")

That is the whole loop. The user says "what database did we land on," recall pulls the one memory about Postgres out of a store that might hold fifty facts, and the model answers as if it remembered, while your context window stays tiny and your token bill stays flat no matter how long the relationship runs. This is the same insight behind fixing retrieval instead of blaming the model: the model was never the problem, the question is what you put in front of it.

Step 4: consolidate, or watch it rot

Here is the step teams skip and regret. People change their minds. A user who told you in March they were building on React tells you in July they moved to Svelte. If you only ever append, your store now holds both, retrieval returns both, and the model gets a contradiction it has to guess its way through. Over months, a naive append-only memory does not get smarter. It gets more confused.

Consolidation reconciles a new fact against what you already hold for that subject before saving.

def consolidate(user_id: str, new_fact: str) -> None:
    """Decide whether a new fact adds to, replaces, or duplicates memory."""
    related = recall(user_id, new_fact, k=5)
    if not related:
        save_memories(user_id, [new_fact])
        return
    resp = client.messages.create(
        model=MODEL,
        max_tokens=300,
        system=(
            "Given existing memories and a new fact, decide the action. "
            'Reply with JSON: {"action": "ADD"|"UPDATE"|"IGNORE", '
            '"replaces": [exact existing memory strings to remove]}. '
            "UPDATE when the new fact supersedes an old one (remove the stale "
            "one). IGNORE if already known. ADD if it is genuinely new."
        ),
        messages=[{
            "role": "user",
            "content": f"Existing:\n{json.dumps(related)}\n\nNew fact: {new_fact}",
        }],
    )
    text = "".join(b.text for b in resp.content if b.type == "text")
    try:
        decision = json.loads(text)
    except json.JSONDecodeError:
        return
    if decision.get("action") == "IGNORE":
        return
    stale = set(decision.get("replaces", []))
    if stale:
        MEMORY_STORE[user_id] = [
            (f, v) for (f, v) in MEMORY_STORE.get(user_id, []) if f not in stale
        ]
    save_memories(user_id, [new_fact])

It is an extra model call per fact, which is not free, but the alternative is a memory store whose quality decays the longer a user stays, which is the exact opposite of the property you were trying to build.

The tradeoffs nobody mentions

Extraction is a quality gamble, and it runs unsupervised. Every fact in your store was written by a model deciding what mattered, with no human in the loop. A bad extraction, a hallucinated preference, a misread of sarcasm, becomes a durable "fact" the agent treats as true for months. Keep the extractor's instructions strict, prefer recall precision over volume, and log what gets written so you can audit it. A small store of correct facts beats a large store you cannot trust.

Memory is user data, with everything that implies. The moment you persist facts about people, you inherit the whole compliance surface: storage security, per-user scoping, and the right to delete. When a user asks to be forgotten, you need to actually purge their memories, not just hide them, and you need to be able to prove you did. Build the delete path on day one. Retrofitting it after you have a million memories is genuinely painful.

More retrieval calls, more latency and cost. Every turn now carries an embedding call and a vector search before the model even starts, and every consolidation is another model call. On a chatty agent this adds up. Batch extraction at the end of a session instead of after every message, cache embeddings, and keep k small. Memory is worth paying for, but pay for it deliberately.

This is retrieval, and retrieval has failure modes. Semantic search over memories misses things exact-match would catch and surfaces things that are merely similar. If a user asks about "the database" and your stored fact says "we chose Postgres," embeddings will usually bridge that, but not always. The same hybrid search and reranking techniques that fix document retrieval apply here once your store grows past a few dozen facts per user.

Do not confuse this with context compaction. Memory across sessions and compaction within a session solve different problems and you often need both. Compaction keeps a single long-running session from overflowing its window. Memory carries the important residue of that session into the next one. Building one does not give you the other.

The takeaway

Statelessness is not a limitation to complain about, it is the boundary you design around. The model will never remember on its own. Memory is a layer you build: extract the durable facts, store them as embeddings scoped to the user, retrieve only the few that matter for the turn in front of you, and consolidate so the store stays true as people change.

Done right, the effect on the user is that the agent simply knows them, without you paying to drag an ever-growing transcript through every call or watching quality sag as the window fills. Start with semantic facts, get extraction and consolidation solid, and add episodic and procedural memory only when a real use case demands them. Most products never need to go further than a few dozen well-chosen facts per user, retrieved three at a time.

If your agent re-introduces itself to your best users every session, or your token bill climbs with engagement instead of falling, there is a memory layer missing from your stack. Book a consultation call and we will design one that fits how your users actually come back.