If you have used an AI tool for a long session, you have probably felt it start to slip: it forgets something you said earlier, loses track of a decision, or repeats itself. That is not the model being dim. It is running into the context window, the single most important constraint to understand about how these tools behave.

What a context window is #

A language model can only consider so much text at one time. That maximum is its context window, and everything the model is working with has to fit inside it: your instructions, the entire conversation so far, any files or data it has been given, and the results of every tool it has used.

The right mental image is a desk of a fixed size. You can spread out only so many papers before you run out of room. To put a new page down, something else has to come off. The context window is that desk, and it does not get bigger mid-task.

Tokens, briefly #

Context windows are measured in tokens, not words. A token is a chunk of text, very roughly three-quarters of a word on average, though it varies with the content (code and non-English text tend to use more tokens per character). “Hello there” might be two or three tokens. A long document might be thousands. When you hear a model has, say, a 200,000-token context window, that is the size of the desk: about 150,000 words of everything, combined, at once.

You do not need to count tokens by hand. The thing to internalize is just that there is a fixed budget, it is shared by everything, and it runs out.

Everything competes for the same budget #

This is the part that explains most of the surprising behavior. That one fixed budget is shared:

• The system instructions that set up the AI’s behavior.
• The full back-and-forth of your conversation, every message, yours and its.
• Any documents, code, or data you have handed it.
• Every tool call an agent has made and every result those tools returned.

In a long agent run, that last item is the quiet killer. An agent that has read twenty files and run thirty commands is carrying all of that output around in its context. The useful signal (your actual goal) is still in there, but it is now buried under a mountain of old tool results competing for the same space.

Why the agent “forgets” #

When the desk fills up, something has to give, and that is when you see forgetting. Depending on the tool, the oldest material gets pushed off to make room, or the session simply cannot continue without intervention. Either way, details from early in the conversation, an instruction you gave at the start, a decision made an hour ago, can fall out of view. The model is not ignoring you. That information is literally no longer on the desk.

This is also why very long single conversations tend to degrade. It is not that the model gets tired. It is that the ratio of relevant context to accumulated clutter keeps getting worse.

How the problem gets managed #

The good news is that this is a well-understood constraint, and there are real techniques for working around it. If you have wondered how AI agents run for hours without falling apart, this is how:

• Compaction (summarizing). When the context gets full, the system replaces a big chunk of the old conversation with a concise summary of it. The details are gone, but the gist, the decisions, and the current state are preserved in far fewer tokens. It is like clearing the desk but keeping a one-page set of notes.
• Pruning (clearing stale material). Old tool results that are no longer relevant get removed outright. The forty-file-reads-ago output is not needed anymore, so it is cleared to free up room, while the conversation structure stays intact.
• External memory. The agent writes important things to a file or an external store, outside the context window entirely, and reads them back only when needed. This is how an agent remembers across sessions, not just within one: the memory lives on disk, not on the desk.

Between these, a well-built agent keeps the desk clear of clutter while holding on to what matters, which is what lets it work on long tasks without losing the thread.

What this means for you #

A couple of practical takeaways fall out of understanding the context window:

• For long tasks, front-load the important stuff and restate it if a session drags on. Something said once at the very start can get pushed off the desk; a clear goal repeated when needed survives.
• Fresh sessions are not a failure, they are a tool. If a conversation has gone on so long that the AI is clearly losing track, starting clean with a tight summary of where things stand often works better than fighting a cluttered context.
• Bigger context windows help but do not eliminate the problem. A larger desk holds more, but any desk fills up eventually, and more clutter still crowds out signal.

The context window is the invisible boundary around everything an AI does. Once you can see it, a lot of otherwise baffling behavior (the forgetting, the repetition, the slow drift on long tasks) turns out to have one simple explanation: the desk got full. It pairs naturally with the agent loop, because every pass of that loop adds more to the very context this article is about.