Context Engineering
Context engineering is the practice of choosing, structuring, and maintaining everything a language model sees when it generates a response: instructions, tools, retrieved documents, conversation history, and memory. Anthropic defines it as the set of strategies for curating and maintaining the optimal set of tokens during inference. Prompt engineering asks how to word an instruction. Context engineering asks what information belongs in the context window at this step, and what should be left out. It matters most for AI agents, which build up context across many turns and tool calls.
Deciding what an AI model gets to see at each step, not just how you phrase the ask.
Category: AI Fundamentals
Where the term came from
The phrase took off in June 2025. Shopify CEO Tobi Lütke said he preferred it to prompt engineering because it better captures the core skill of "providing all the context for the task to be plausibly solvable by the LLM." Andrej Karpathy agreed, describing the work behind serious LLM applications as "the delicate art and science of filling the context window with just the right information for the next step." LangChain published a framework for it in July 2025, and Anthropic published its engineering guide on September 29, 2025.
The rename stuck partly because "prompt engineering" had drifted in everyday use. As Simon Willison noted at the time, most people heard it as typing things into a chatbot. "Context engineering" points at the real job: building a system that assembles the right inputs automatically, on every step.
Context engineering vs prompt engineering
| Dimension | Prompt engineering | Context engineering |
|---|---|---|
| Main question | How should I word this instruction? | What should the model see right now? |
| Scope | Mostly the system prompt and user prompt | The whole context window: prompt, tools, data, history, memory |
| Timing | Usually written once, then revised | Runs again every time information is passed to the model |
| Typical owner | Anyone writing prompts | Engineers building the application or agent |
| Typical failure | Vague or ambiguous instructions | Missing, stale, conflicting, or excess information |
One does not replace the other. Anthropic calls context engineering the natural progression of prompt engineering, and a clear system prompt is still part of the job. A beautifully worded prompt will not help if the agent never retrieved the document it needed.
The four core strategies
LangChain's write, select, compress, and isolate framework is the most widely used map of the techniques. Anthropic's guide describes matching methods under different names.
| Strategy | What it means | Common technique |
|---|---|---|
| Write | Save information outside the context window for later | Scratchpads, a NOTES.md file, long-term memory |
| Select | Pull in only what the current step needs | RAG, memory lookup, just-in-time file reads, tool search |
| Compress | Keep the meaning and drop the tokens | Conversation summaries, compaction, clearing old tool results |
| Isolate | Split work so each part sees only what it needs | Sub-agents with clean windows, sandboxes, separate state fields |
Anthropic's guiding target is the "smallest set of high-signal tokens" that maximizes the chance of the outcome you want. It also recommends just-in-time retrieval: the agent holds lightweight references such as file paths, then loads the content with tools only when needed, often alongside some data retrieved up front.
Worked example: shrinking one agent turn
Take a study agent answering a question about one chapter of a course. These token counts are illustrative, not measurements. They show how the four strategies change what a single request costs.
| Layer | Naive build | Tokens | Engineered build | Tokens |
|---|---|---|---|---|
| System prompt | Long rule list | 3,000 | Short sections plus a few examples | 2,000 |
| Tools | 40 definitions always loaded | 25,000 | 8 tools, found on demand | 4,000 |
| History | Every prior turn | 60,000 | Summary plus last 5 turns | 14,000 |
| Documents | The whole 40-page PDF section | 120,000 | Six retrieved chunks | 6,000 |
| Total | 208,000 | 26,000 |
At Claude Sonnet 5.5's list price of $2 per million input tokens, the naive request costs about $0.42 in input and the engineered one about $0.05. Both fit in a 1M-token window, which is the point: the window size tells you what is allowed, not what is wise. Anthropic's docs warn that accuracy and recall degrade as token count grows, a problem they call context rot.
def build_context(question, history, retriever, summarize, max_chunks=6):
"""Assemble only what this step needs. Function names are illustrative."""
chunks = retriever.search(question, top_k=max_chunks) # select
older, recent = history[:-5], history[-5:]
summary = summarize(older) if older else "" # compress
return [
{"role": "system", "content": SYSTEM_PROMPT}, # stable prefix first
{"role": "user", "content": (
f"Summary of earlier conversation:\n{summary}\n\n"
f"Relevant passages:\n{format_chunks(chunks)}"
)},
*recent,
{"role": "user", "content": question}, # the task goes last
]How context goes wrong
Drew Breunig's 2025 post named four failure modes that are now common vocabulary among agent builders:
| Failure | What happens | Typical fix |
|---|---|---|
| Context poisoning | A hallucination or error enters the context and keeps being referenced | Validate before writing to memory; restart from a clean state |
| Context distraction | A very long context pulls attention to accumulated history | Summarize or compact older turns |
| Context confusion | Superfluous content, such as unused tools, lowers response quality | Load fewer tools; retrieve them on demand |
| Context clash | Pieces of information in the context contradict each other | Remove stale data; split work between sub-agents |
Context is also an attack surface: Anything your agent retrieves can carry instructions. Web pages, emails, and uploaded files can all deliver prompt injection, so a context pipeline should treat retrieved text as data, not as commands.
Honest caveats
- Much of this is not new. RAG, summarization, and memory existed before the name did. The term groups them into one discipline, which is useful, but it does not describe a new algorithm.
- Less context is not always better. Anthropic warns that overly aggressive compaction can lose subtle but critical context. It advises tuning summaries on complex agent traces, maximizing recall first and then trimming.
- No single metric proves it works. Token counts are easy to measure, but task success is what counts. Test changes with agent evaluation rather than assuming a smaller context is an improvement.
- Vendor advice ages quickly. Features such as server-side compaction and tool search change between model generations, so check current docs before you design around one.
Frequently asked questions
Is context engineering replacing prompt engineering? No. It widens the scope. You still need clear instructions, but they are now one part of a larger system that decides what the model sees.
Do I need context engineering for ordinary chatbot use? Only in a light form. Starting a fresh chat for a new topic, attaching just the relevant pages, and pasting your constraints are all small acts of context engineering.
How does it differ from RAG? RAG is one technique, and it sits under the select strategy. Context engineering also covers memory, tool loading, summarization, and sub-agent design.
Does a 1M-token window make this unnecessary? No. A larger window delays hard limits, but it does not remove cost, latency, or context rot. See context window for the mechanics.
How does prompt caching fit in? Stable content placed first in the context can be reused cheaply through prompt caching. Cached tokens still occupy the window, so caching lowers the bill without solving context rot.
You can practice context engineering in any LumiChats chat: start a new conversation for each topic, upload only the pages you need, and compare how the same trimmed context performs across models.