Context Engineering for AI Agents: Concepts, Failure Modes, and Core Strategies
The Foundations of Context Engineering for AI Agents
Context engineering is the discipline of controlling what information an LLM sees at each step of an agent’s execution. Agents may have access to system instructions, conversation history, retrieved knowledge, tool definitions and outputs, scratchpads, memories, plans, execution state, and other artifacts.
The challenge is not simply fitting this information into a token budget, but deciding what should be written outside the context window, what should be selected back in, what should be compressed, and what should remain isolated.
Poorly managed context can degrade agent behavior through context poisoning, distraction, confusion, and clash. These problems become more important as agents operate over longer trajectories, use more tools, and maintain state across sessions.
This blog first introduces context engineering, explains why it matters, and breaks down the main types of information that enter an agent’s context, including instructions, knowledge, tools, memory, and state. It then examines what can go wrong when that context is poorly managed, including context poisoning, distraction, confusion, and clash.
Finally, it presents four practical strategies for managing context throughout an agent’s trajectory: writing important information outside the context window, selecting only what is relevant, compressing accumulated context through summarization and trimming, and isolating information in state, execution environments, or separate agents until it is needed.
Table of Contents:
Introduction to Context Engineering
Why do we need Context Engineering?
What goes into an agent’s context?
What Happens When Context Goes Wrong?
How to tackle these problems?
5.1. Write Context
5.2. Select Context
5.3. Compressing Context
5.4. Isolating Context
Want to Go Deeper into Context Engineering?
I am hosting a 3-hour live workshop on Context Engineering. You will learn what context engineering really means in practice, and why it’s one of the most important factors shaping the quality of agents’ outputs.
Date: Sunday, 30 August · 18:00–21:00 EEST
Early-bird price is $35 with CONTEXT35 until 16 August at 23:59 EEST. After that, the original price is $50.
1. Introduction to Context Engineering
Context engineering is the art and science of filling the context window with just the right information at each step of an agent’s trajectory. LLM is like the CPU, and its context window is like the RAM, serving as the model’s working memory.
Just like RAM, the LLM context window has a limited capacity to handle various sources of context. And just as an operating system curates what fits into a CPU’s RAM, we can think about “context engineering” playing a similar role.
“Context engineering” … is effectively the #1 job of engineers building AI agents.” by Cognition
The important limitation is that this working memory is finite. Even when a model supports a very large context window, simply filling it with more information does not guarantee better results. Irrelevant, duplicated, outdated, or poorly structured context can make it harder for the model to identify what matters.
This is where context engineering becomes important. The system has to continuously decide:
what information should be added,
what should be retrieved only when needed,
what should be compressed or summarized,
what should remain outside the context window,
and what should be removed when it is no longer useful.
For an agent, these decisions happen repeatedly throughout its trajectory. The right context for the first planning step may be very different from the context needed after several tool calls, retrieved documents, and intermediate results.
So context engineering is not just about fitting information into a token budget. It is about maintaining the most useful working state for the model at every step of the task.
2. Why do we need Context Engineering?
An LLM can only reason over the information available in its current context. A document stored in a database, a memory saved from a previous session, or the result of an earlier tool call is effectively invisible to the model unless the application brings that information into the context window.
In simple applications, assembling that context is relatively straightforward. A user provides a prompt, the application adds a system instruction, and the model generates a response. Once we move to agents, however, the amount of potentially useful context grows quickly.
During a single task, an agent may accumulate instructions, conversation history, retrieved documents, tool definitions, tool outputs, plans, intermediate results, memories, errors, and observations from the environment. After only a few steps, there may be far more information available than the model needs for its next decision.
The obvious solution might seem to be: put everything into the context window. But a larger context window does not eliminate the context-engineering problem.
First, context windows still have finite capacity. More importantly, models do not necessarily use every token in a long context equally well. Research on the lost-in-the-middle phenomenon, for example, has shown that model performance can depend on where relevant information appears inside a long context.
There are also practical problems beyond the token limit. Irrelevant information competes with useful information. Old observations can become stale. Different sources may contradict each other. Large tool outputs can consume thousands of tokens even when only a few lines are useful. Repeatedly passing all of this information back to the model also increases latency and inference cost.
So the objective is not to maximize the amount of context. The objective is to construct the smallest useful context that gives the model enough information to make the next decision correctly.
This becomes especially important for agents because context changes throughout the agent’s trajectory. The information needed to create a plan may be different from what is needed to execute a tool, inspect its result, recover from an error, or produce the final answer.
3. What goes into an agent’s context?
Most LLM applications construct context from several categories of information.
3.1. Instructions
Instructions define how the model should behave and what it is expected to do. They can include:
System and developer prompts
Task-specific instructions
Policies and constraints
Few-shot examples
Output-format requirements
Tool descriptions and schemas
Procedural memories or reusable instructions
For agents, instructions are particularly important because they influence both reasoning and action selection. OpenAI, for example, describes instructions, tools, and the model itself as core components of an agent.
3.2. Knowledge
Knowledge provides the information the model needs to reason about the task. Unlike instructions, it usually describes what is true or relevant, rather than what the model should do. Examples include:
Retrieved documents
Database records
Search results
Knowledge-base passages
User or organizational memory
Domain-specific facts
Information extracted from files
A RAG system is one example of context engineering at this layer: rather than placing an entire knowledge base into the prompt, the system retrieves a small subset of relevant information for the current request.
3.3. Tool feedback
When an agent interacts with external systems, those systems produce new context. For example:
search(query) → search results
read_file(path) → file contents
run_tests() → test output
query_database() → rows
execute_code() → stdout, errors, artifacts
These observations are often essential for deciding what the agent should do next. But raw tool outputs can also become one of the largest sources of context growth. A command might return 5,000 lines even though only one error message matters.
Context engineering therefore determines not only which tool to call, but also how much of its result should remain available afterward.
3.4. Agent state and history
Agents operate across multiple steps, so they need some representation of what has already happened. This can include:
Conversation history
The current plan
Completed and pending tasks
Intermediate results
Previous decisions
Execution state
Scratchpad information
Summaries of earlier steps
Without persistent state, the agent may repeatedly rediscover information or lose track of its progress. But keeping the entire trajectory verbatim creates the opposite problem: the context becomes increasingly dominated by historical information.
Context engineering therefore has to decide what state should remain active, what should be summarized, and what can be moved outside the context window.
As an agent moves through a task, these four sources continuously compete for space and attention inside the context window. Context engineering is the set of mechanisms we use to control that flow: deciding what to write into context, what to retrieve, what to compress, and what to keep isolated outside the model’s working memory.
4. What Happens When Context Goes Wrong?
The four context sources we discussed earlier: instructions, knowledge, tools, and state/history, are all useful. The difficulty appears as an agent moves through a task and continuously adds information from each source.
A retrieval step may add several documents. A tool call may return hundreds or thousands of tokens. The agent may update its plan, save an observation, retrieve a memory, call another tool, and append the result to the conversation. Over a long trajectory, the context window can gradually turn into an append-only log of everything that has happened.
At that point, reaching the model’s maximum token limit is only one possible failure. Long-running agents can become slower and more expensive as context grows, but more importantly, the quality of the context itself can degrade the agent’s behavior even when the context still fits comfortably inside the model’s window.
LangChain groups several of these failure modes into context poisoning, distraction, confusion, and clash, based on a taxonomy described by Drew Breunig. These failures are especially important for agents because an error at one step does not necessarily disappear. The output of one model invocation often becomes part of the input to the next invocation, creating feedback loops where bad context can persist and compound.
1. Context Poisoning
Context poisoning happens when incorrect information enters the context and is subsequently treated as if it were reliable.
