Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) combines information retrieval with generative inference. An application retrieves information relevant to a user's request, includes that information in the model's input context, and asks the model to produce a response grounded in the retrieved evidence. The retrieval step does not, by itself, change the model's weights. This distinction matters when choosing between RAG, prompt engineering, and fine-tuning: RAG supplies external knowledge at request time, while fine-tuning changes model parameters to influence behavior or task performance.

A typical document-based implementation has ingestion and query stages. During ingestion, documents are parsed, divided into chunks, and indexed with metadata such as document identifiers, update timestamps, and access restrictions. Many systems create vector embeddings that represent semantic meaning. At query time, the application searches for relevant chunks using vector similarity, keyword search, or a hybrid approach. It may rerank results before supplying selected passages to the generation model. RAG does not require a vector database; retrieval can also use conventional search, databases, or other knowledge sources.

In the cloud, object storage can hold source files, a search service or database can maintain an index, and a managed model service can provide embeddings and text generation. Managed capabilities such as Amazon Bedrock Knowledge Bases can coordinate ingestion, retrieval, and generation. Engineers still need to choose source data, configure retrieval behavior, enforce permissions, and evaluate the resulting answers. Updating a source document does not necessarily update its indexed representation immediately, so ingestion schedules and deletion propagation affect freshness.

Business applications include employee policy assistants, customer-support tools, technical documentation search, and product-selection assistants. For example, a support application can retrieve troubleshooting instructions for a specific device model and firmware version, then generate an explanation with references to the relevant manual. A policy assistant can retrieve the policy applicable to an employee's location instead of relying on general information learned during model training.

RAG is an important part of context engineering: deliberately selecting and organizing the instructions, evidence, conversation history, and tool outputs available to a model. Chunk size, metadata filters, query rewriting, reranking, and context ordering all affect whether the model receives useful evidence. Adding more documents is not automatically better. Irrelevant or conflicting passages consume context capacity, increase token costs, and can distract the model. Instructions should specify how to handle insufficient evidence, conflicting sources, and requests for citations.

Evaluate retrieval and generation separately, then test the complete application. Retrieval evaluation can measure whether relevant evidence appears among the top results, using metrics such as recall at k and ranking quality. Generation evaluation should examine correctness, relevance, completeness, faithfulness to retrieved evidence, and whether citations actually support the claims. Use representative questions, expected answers, and reference passages, including unanswerable questions and permission-sensitive cases. Human review and calibrated model-based judges can complement automated checks. For agents and workflows, also evaluate tool selection, intermediate steps, and final task completion rather than scoring only the last response.

RAG reduces some causes of hallucination but does not guarantee factual answers. A model can misinterpret evidence, invent unsupported details, or cite an irrelevant passage. Retrieved documents can also contain prompt-injection instructions, so treat their contents as untrusted data rather than authoritative commands. Apply document-level authorization before retrieved content reaches the model, protect sensitive context, and monitor latency, cost, retrieval failures, and answer quality. RAG introduces search and indexing overhead, but it can make knowledge updates more manageable than repeatedly retraining a model.