In-Context Learning

In-context learning changes how a foundation model performs a task by changing its input rather than its parameters. The application supplies instructions, demonstrations, reference information, or conversation history, and the model uses that context when generating its response. This adaptation does not create a newly trained model or permanently teach the model the supplied facts. To reuse the same guidance in a later request, the application generally needs to supply it again through the prompt or managed conversation state.

Prompt engineering controls how this context is presented. Zero-shot prompting describes a task without providing demonstrations, such as asking a model to classify a support ticket into a named category. Single-shot, also called one-shot, prompting supplies one example of the desired input and output. Few-shot prompting supplies several examples so the model can infer label meanings, formatting conventions, or decision patterns. Although in-context learning is often used broadly for prompt-based adaptation, its narrower research usage commonly emphasizes learning a task from demonstrations.

For example, a support application can provide three labeled tickets, define the categories billing, access, and outage, and then ask the model to classify a new ticket. Examples should cover meaningful distinctions rather than repeat nearly identical cases. A prompt template can standardize the instructions, examples, ticket placeholder, and required output schema. The template is a reusable construction mechanism, not a training method, and the application should validate generated output before using it to route work.

Chain-of-thought prompting is a separate technique that encourages intermediate reasoning, sometimes through worked examples. It can overlap with few-shot prompting when demonstrations include reasoning, but it is not synonymous with in-context learning. A long explanation does not establish that an answer is correct. In production, request concise justifications or verifiable intermediate results when useful, and check calculations, tool outputs, and business constraints independently.

In cloud applications, an orchestration layer typically assembles the prompt, selects relevant examples, attaches authorized context, and calls a managed model API or hosted model endpoint. This supports adaptability and conversational responsiveness: the same model can draft a customer reply, extract invoice fields, or summarize an incident using different context. Conversation history can help maintain continuity, but only history actually included or otherwise made available to the model can influence the current response.

In-context learning usually has lower setup cost than training because it requires no weight-update job. However, demonstrations consume input tokens on repeated requests, increasing cost and often latency. Pre-training a model from scratch generally requires far more data, compute, and engineering. Fine-tuning adds training, evaluation, and model-management costs, but may allow shorter prompts for stable, frequently repeated tasks; total savings depend on traffic, model pricing, and deployment costs. Model distillation trains a smaller student to reproduce useful teacher behavior, trading additional development and training work for potentially lower serving cost.

Retrieval-augmented generation, or RAG, retrieves relevant external information and places it in the model’s context. It can complement in-context learning, but retrieval and learning from demonstrations are not identical operations. RAG introduces document ingestion, embedding where applicable, indexing, retrieval, and storage costs, while reducing the need to place an entire knowledge collection in every prompt. It is often appropriate for changing business facts; fine-tuning is not a dependable substitute for a current, authoritative knowledge source.

Practical limits include finite context windows, sensitivity to example selection and ordering, hallucinations, and inconsistent adherence to instructions. More examples do not necessarily improve accuracy, and important details can be missed even when they fit within the context window. Treat retrieved documents and user submissions as untrusted data, enforce access controls before adding content, and test for prompt injection. Evaluate representative and difficult cases, measure token usage and latency, and choose the smallest reliable context rather than the largest available one.