Query Decomposition

Query decomposition is a query-handling technique that divides a complex question into smaller, answerable subqueries. In retrieval-augmented generation (RAG), its purpose is to obtain the evidence a foundation model needs before generating an answer. It is especially useful for comparisons, questions spanning multiple documents, and requests in which one fact must be discovered before another can be retrieved. The same principle appears in database query planning, but decomposing a natural-language request for retrieval does not necessarily involve generating SQL or optimizing database execution plans.

Consider an engineer asking, “How do the production and development backup policies differ, and which production requirements does our current configuration fail to meet?” A single semantic search might retrieve a relevant policy but miss the configuration evidence. A decomposition pipeline can retrieve the production policy, the development policy, and the current production configuration separately. It can then compare the policies and evaluate the configuration against the production requirements, citing the evidence for each finding.

Subqueries can be independent or dependent. Independent requests can run concurrently to reduce elapsed time. Dependent requests must run in sequence: a question about the retention policy for the database used by a particular application first requires identifying that database. Representing these dependencies explicitly prevents the system from guessing intermediate facts. Each subquery should retain necessary entities, dates, environments, and other constraints from the original request.

In a cloud implementation, application code or a foundation model produces a bounded, structured query plan. A function validates that plan, and a workflow orchestrator schedules retrieval, handles failures, and collects results. On AWS, an application can invoke a model through Amazon Bedrock to propose subqueries, use AWS Lambda for validation and retrieval logic, and use AWS Step Functions to coordinate transformations and parallel or sequential execution. Retrieval can target a knowledge base or a search index, depending on the application architecture.

Decomposition is different from query expansion. Expansion adds synonyms, related terms, or alternative formulations to improve retrieval coverage; decomposition separates distinct information needs. Query transformation is the broader category that can include both. A pipeline may expand each decomposed subquery, but issuing several paraphrases of the same question is not necessarily decomposition.

After retrieval, the application should deduplicate overlapping passages, preserve source identifiers, and select or rerank evidence within a context budget. Retrieval scores from different queries are not automatically comparable. Final synthesis must address the original question, distinguish conflicting sources, and identify unsupported portions rather than filling gaps with plausible claims. Summarizing each branch too aggressively can discard qualifications needed for the final comparison.

The main trade-offs are additional model calls, retrieval requests, latency, and orchestration complexity. Excessive splitting can remove useful context or amplify an incorrect interpretation. Set limits on subquery count, dependency depth, retries, concurrency, and token usage, and bypass decomposition for simple questions. Apply authorization and tenant filters to every retrieval branch; model-generated instructions must not broaden access. Evaluate against a single-query baseline using evidence coverage, answer correctness, citation support, latency, and cost, including cases with missing evidence or failed branches.