Context engineering
Choosing what goes into the model's context window, in what order, and what to leave out, because the model sees nothing else.
Read first: Tokens & tokenization, How a model predicts the next token.
D1 · StoryFor young children and total beginners
The AI can only use what is in front of it right now. That space is called its context, and it has a size limit.
Context engineering is deciding what to put in that space: the instructions, the right notes and the question, in a sensible order.
Like: Packing a small school bag for the day.
D2 · PictureFor curious beginners
Every model call is built from parts: a system prompt with rules and role, tool descriptions, retrieved documents, memory, the conversation so far and the latest message.
All of it must fit in the context window, which is measured in tokens. Long contexts cost more, run slower, and models can miss details buried in the middle.
So the application budgets space, trims or summarises old turns, and keeps stable parts (system prompt, tool list) at the start so a cached prefix can be reused.
Like: Packing a small school bag for the day.
Where it breaks: A bag holds things you can dig out later. A model has no bag outside the window: anything left out simply does not exist for that call, and stuffing in more can make it notice less.
D5 · FrontierFor experts
A bigger context window does not mean the model uses all of it well.
Uneven use. Models used information at the start and end of a long context better than information in the middle (Lost in the Middle, 2023).
Effective length is shorter than advertised. Of 17 models claiming 32,000 tokens or more, only about half performed well at 32,000 (RULER, 2024); in 2025, 18 models grew less reliable as input lengthened, even on simple tasks (Context Rot, 2025).
Compaction. Long agent sessions have to be summarised, and what a summary drops can matter later. Compaction, note-taking and sub-agents help but are still tuned by hand (Anthropic, 2025).
Untrusted text in context. Anything placed in the context can carry instructions, so choosing what goes in is also a security decision (Greshake et al., 2023).
Examples as of 2026-10-30
- Agent instruction files such as CLAUDE.md and AGENTS.md
- Context compaction in coding agents
- Agent Skills (instructions loaded only when needed)
- Prompt caching for long, stable context
Sources
- Effective context engineering for AI agents (Anthropic engineering blog) (2025-09-29)
- Lost in the Middle: How Language Models Use Long Contexts (2023)
- RULER: What's the Real Context Size of Your Long-Context Language Models? (2024-04-09)
- Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma) (2025-07-14)
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023)
How it connects
- Retrieval-augmented generation (RAG) supplies passages to Context engineering.
- Tool use & protocols (MCP) returns results into Context engineering.
- Memory reloads saved facts into Context engineering.
- Guardrails & evals scores changes to Context engineering.
- Batching & KV caching reuses stable prefixes in Context engineering.
More in Layer 7 · Orchestration & Tooling
- Retrieval-augmented generation (RAG)
Look up relevant documents first, then give them to the model to answer from. - Tool use & protocols (MCP)
Letting a model call functions and services through structured requests, standardised by protocols such as MCP. - Memory
How an AI application remembers beyond one call: short-term memory in the context window, long-term memory in stores it saves to and searches. - Guardrails & evals
Checks around model calls (input and output filters, permission gates, prompt-injection defences) plus automated tests that measure whether the system still behaves.