A context window is the finite capacity within which a foundation model processes information during inference. For language models, this capacity is usually expressed in tokens rather than words or characters. Tokens can represent words, word fragments, punctuation, or other encoded content. Multimodal models also consume capacity for inputs such as images and audio, using model-specific accounting rules. Context limits differ across models and versions, so applications must use the limits and token-counting behavior of their selected model.
The context supplied to a model can include system instructions, the current question, earlier conversation turns, examples, retrieved passages, tool definitions, and tool results. Many models have a combined input-and-output context limit, along with a separate maximum output length. Applications must reserve sufficient capacity for the generated response and check the provider's specific rules. For example, with a hypothetical 32,000-token combined limit, allocating 28,000 tokens to input leaves at most 4,000 tokens for generation, subject to any lower output limit.
Context engineering is the broader practice of deciding what information enters this limited space, how it is structured, and when it is refreshed. Prompt engineering focuses primarily on instructions and examples; context engineering also addresses retrieval, conversation state, tool outputs, and application data. In a cloud application, an orchestration layer can assemble these inputs before calling a hosted foundation model. It should count or estimate tokens, enforce budgets, and handle oversized requests explicitly rather than assuming the endpoint will truncate them safely.
Consider an internal support assistant answering a question about a database backup policy. Instead of placing an entire operations library into the prompt, the application retrieves a small set of relevant passages, includes their document identifiers, and asks the model to answer using that evidence. This retrieval-augmented generation approach supplies current, task-specific information without changing model weights. Retrieval does not expand the context window: the selected passages still occupy space within it.
Conversation history creates a similar challenge. A long-running troubleshooting session may exceed the model's capacity even when each individual message is short. The application can retain recent exchanges, preserve essential facts in a structured summary, and retrieve older details when needed. Summarization saves space but can omit constraints or introduce errors, so important facts such as resource identifiers and approved actions should be preserved and validated. Stored conversation history is not automatically available to the model; the application or service must supply the relevant portions.
Larger windows can accommodate more evidence, but they do not guarantee better reasoning or factual accuracy. Irrelevant passages, contradictory instructions, and poorly positioned evidence can reduce answer quality. More input commonly increases token charges and processing time, although pricing, caching, and latency behavior vary by model and service. Engineers should evaluate retrieval relevance, answer correctness, token consumption, and response latency together rather than choosing a model solely for its advertised window size.
Context boundaries are also security boundaries. Retrieved documents and tool responses can contain malicious instructions, so applications should treat them as untrusted data, distinguish evidence from authoritative instructions, and enforce authorization before retrieval. Sensitive information should be minimized before it reaches the model. For certification scenarios, distinguish context engineering from fine-tuning: context engineering changes the information available for a particular request, whereas fine-tuning changes model parameters through additional training. Neither a large context window nor retrieval alone eliminates hallucinations.
Amazon OpenSearch Service