Cloud concept · Updated

Quality Gates - Quality gates are explicit pass-or-fail checkpoints that use evidence to determine whether a change can advance through a delivery workflow.

In short

A quality gate turns testing, evaluation, and operational requirements into enforceable promotion criteria. In cloud delivery, gates prevent unacceptable application, infrastructure, or generative AI changes from reaching broader environments while providing an auditable explanation for each release decision.

Cloud Concepts Hub →

6 key aspects of Quality Gates

Effective quality gates combine measurable standards, appropriately staged tests, trustworthy evidence, and enforceable promotion or recovery decisions.

01
Enforceable acceptance criteria

Translate requirements into explicit thresholds and blocking conditions. Define what happens when evidence fails, is missing, or is too old rather than treating report generation as successful validation.

02
Stage-appropriate testing

Run fast, deterministic checks early and use deployed environments for integration, end-to-end, security, and performance testing. Match test scope and execution cost to the risk at each promotion boundary.

03
Systematic GenAI regression evaluation

Evaluate model, prompt, retrieval, and configuration changes against versioned representative and adversarial datasets. Compare both absolute standards and regressions against an approved baseline, including results for critical categories.

04
Reliable evaluation evidence

Account for stochastic outputs, evaluator limitations, and flaky tests. Use repeated measurements, calibrated rubrics, and targeted human review where automated scores alone are insufficient.

05
Progressive deployment safeguards

Observe canary or blue/green deployments before expanding exposure. Connect failed operational criteria to rollout termination and configured rollback rather than relying only on deployment completion.

06
Traceability and controlled exceptions

Associate decisions with immutable artifacts and versioned tests, datasets, evaluators, and policies. Require explicit authorization and expiration for overrides so release evidence remains auditable.

Cloud services for Quality Gates

AWS CodePipeline establishes promotion boundaries, while AWS CodeBuild executes tests and converts acceptance-policy violations into failed actions. Amazon Bedrock evaluations can provide GenAI quality evidence; custom automation or AWS Step Functions can wait for evaluation completion, assess thresholds, and return a gate decision. During deployment, AWS CodeDeploy can use validation hooks and Amazon CloudWatch alarms to stop unhealthy rollouts and initiate configured rollback. These services provide execution and orchestration mechanisms, but engineers must define the acceptance policy, preserve evidence, and ensure failures actually prevent promotion. Reports and dashboards without an enforced decision path are observability, not quality gates.

AWS CodePipelineAWS CodePipeline

Coordinates delivery stages and prevents downstream promotion when required actions fail. Stage conditions, test actions, and manual approval actions can implement different gate policies, while external evaluation results require an integration that translates them into an enforceable decision.

AWS CodeBuildAWS CodeBuild

Runs unit tests, integration tests, security tools, and custom evaluation scripts in managed build environments. A gate implementation must propagate failing tests or unacceptable scores into build failure; publishing a test report alone does not enforce acceptance criteria.

Amazon Bedrock

Provides model evaluation capabilities that can supply evidence for foundation model and generative AI application quality decisions. Pipeline automation must retrieve evaluation results, compare them with project-specific thresholds, and explicitly block promotion when requirements are not met.

AWS CodeDeployAWS CodeDeploy

Supports deployment validation hooks and alarm-based deployment controls for supported compute platforms and deployment configurations. Configured automatic rollback can recover from deployment failures or alarm conditions, providing a gate during rollout rather than only before deployment.

Amazon CloudWatchAmazon CloudWatch

Supplies metrics, logs, and alarms for operational acceptance criteria such as error rates and latency. Custom metrics can also represent application or GenAI evaluation outcomes, but a deployment or workflow integration must act on those signals.

AWS Step FunctionsAWS Step Functions

Orchestrates custom evaluation workflows that invoke services, wait for asynchronous jobs, compare results, and branch into pass or fail paths. It is useful when a gate requires multiple evaluations, retries, or an explicit review workflow beyond a single build action.

Quality Gates exam questions

Real practice exam questions on Quality Gates, with the correct answer and its explanation. Answer them with every option in the exam simulator.

DVA-C02

A developer is designing a release workflow that deploys an application to staging and then invokes two existing AWS Lambda functions: one performs security validation, and the other performs compatibility validation. Each function returns {"passed": true} or {"passed": false} as a normal, successful invocation. A failed validation does not raise an exception. The validations must run concurrently, and production deployment must start exactly once, only after both validations have completed and both have passed. Which workflow design meets these requirements?

Show answerCORRECT ANSWER

Use an AWS Step Functions Parallel state with one validation task in each branch. After the Parallel state, use a Choice state that requires both returned passed values to be true before transitioning to production deployment. Route other results to a Fail state.

A Parallel state waits for all branches to complete successfully. The subsequent Choice state can evaluate both returned validation results with an And condition. A single production deployment state after this gate runs only when both validations pass.

SAP-C03

A company is redesigning an AWS CodePipeline pipeline for a business-critical application. The build stage already produces the release artifact. Production rollout uses six existing actions: DeployPilot, CheckPilot, ApproveExpansion, DeployFleetA, DeployFleetB, and CheckFleet. Each automated check reports action failure if its acceptance criteria are not met.

The release policy requires the following workflow:
• Deploy the release to the pilot fleet, and then check the pilot fleet's health.
• Request manual approval only after the pilot health check succeeds.
• After approval, deploy to Fleet A and Fleet B in parallel. These fleets are independent.
• Run the final fleet health check only after both fleet deployments succeed.

All six actions must remain in one Production stage so that operators can control entry to the entire production rollout through one stage transition. The company wants CodePipeline to enforce the dependencies without orchestration scripts.

Which action configuration meets these requirements?

Show answerCORRECT ANSWER

Assign runOrder 1 to DeployPilot, 2 to CheckPilot, 3 to ApproveExpansion, 4 to DeployFleetA and DeployFleetB, and 5 to CheckFleet.

Within a CodePipeline stage, actions with the same runOrder can run in parallel, and actions with a higher runOrder wait for the preceding actions to succeed. This configuration enforces pilot deployment, pilot validation, approval, parallel fleet deployments, and final validation in the required sequence.

AIP-C01

A company releases prompt-template updates for a generative AI assistant through AWS CodePipeline. An AWS CodeBuild action evaluates each candidate and returns a nonzero exit code when acceptance criteria are not met. The prompt repository contains the evaluation script, buildspec, and acceptance thresholds.

During an audit, the company discovers that a prompt update passed because its author also lowered the factual-correctness threshold in the same commit. The evaluation results were accurate, and the pipeline correctly enforced the modified threshold.

The company requires that prompt developers cannot weaken or bypass release criteria through changes to the prompt repository. Only the quality assurance team may modify the evaluator or acceptance policy. Candidates that meet the approved criteria must continue to deploy automatically.

Which design best meets these requirements?

Show answerCORRECT ANSWER

Move the evaluator, buildspec, and acceptance policy to a QA-controlled location. Configure a mandatory CodeBuild action to use those protected components to evaluate candidate prompt artifacts and fail when approved criteria are not met. Restrict changes to the gate project, pipeline, and protected components to authorized administrators and QA personnel.

This separates candidate-controlled artifacts from the components that authorize promotion. The mandatory action enforces QA-approved criteria, while access restrictions prevent prompt developers from replacing or bypassing it. Passing candidates can still proceed automatically.

Practice Quality Gates in a full exam

Questions on Quality Gates in each practice exam, with every answer option and an explanation for each:

Start a free practice exam

What is Quality Gates

3 min read

A quality gate evaluates evidence against predefined acceptance criteria before allowing a workflow to proceed. The evidence might include test results, vulnerability findings, performance measurements, foundation model evaluations, or production health metrics. Running tests alone does not create a gate: the workflow must interpret the results and block promotion, request review, or initiate a defined recovery action when requirements are not met.

Place gates where they provide useful feedback at an appropriate cost. Before merging code or during an initial build, run fast unit tests, static analysis, dependency checks, and infrastructure template validation. After deploying to an isolated test environment, run integration and contract tests against real service interfaces, including authentication and authorization behavior. Reserve broader end-to-end, load, resilience, and dynamic security tests for suitable staging environments or scheduled workflows because these tests generally require more time and resources. Critical release requirements must still be satisfied before production promotion; scheduling a test separately does not make its result optional.

For example, an API pipeline might reject a build when unit tests fail or a prohibited critical vulnerability is found. After staging deployment, another gate could require checkout integration tests to pass and measured request latency to remain within an agreed budget under representative load. A production canary can then expose a small traffic share while operational metrics are observed. Failed health criteria should stop further rollout and, where supported and configured, trigger rollback. A deployment completing successfully is not evidence that application behavior is correct.

For generative AI applications, treat changes to the foundation model, prompt templates, retrieval configuration, knowledge corpus, guardrails, and inference parameters as potential behavioral changes. Evaluate candidates against versioned datasets containing representative tasks, difficult cases, known failures, and adversarial inputs. Gate on task-relevant measures such as answer correctness, groundedness, retrieval relevance, unsafe response rates, sensitive-data disclosure, latency, and cost. For example, a support assistant might require minimum grounded-answer performance, no observed disclosure in a designated privacy test suite, and no material regression on critical customer workflows. Passing a finite test suite does not prove that disclosure can never occur.

Model outputs are often nondeterministic, so exact string matching is usually unsuitable for open-ended answers. Combine deterministic checks for schemas and tool arguments with semantic evaluation, rubric-based assessment, and targeted human review. Repeated trials and uncertainty estimates help distinguish meaningful regressions from sampling variation. Model-based judges should be calibrated against human judgments and checked for bias and inconsistency. Do not let a strong average score hide failures in safety-critical categories or important user segments.

Continuous evaluation extends gates beyond the initial release. Run regression suites on proposed changes and periodically evaluate deployed behavior using appropriately protected production samples. Keep an untouched holdout set for unbiased comparisons, and add newly discovered failures to a maintained regression set. Production monitoring can identify distribution shifts or emerging problems and trigger investigation, reevaluation, or rollback; it does not replace predeployment validation.

Define thresholds, evidence freshness, missing-result handling, and exception procedures explicitly. A report that exists but is stale, incomplete, or associated with another artifact should not satisfy the gate. Record the exact artifact, environment, dataset, evaluator, and policy versions so decisions can be reproduced. Promote the same immutable application artifact between environments rather than rebuilding untested binaries.

Strict gates reduce release risk but can increase delivery time, compute cost, and false rejections. Use risk-based test selection, parallel execution, and controlled isolation of flaky tests without silently bypassing essential coverage. Human approval can complement automated evidence for high-impact changes, but approval alone is not proof of quality. Exceptions should be authorized, time-limited, and auditable rather than becoming permanent bypasses.

Learn Quality Gates in lessons

3 short lessons on Quality Gates in 3 certification courses.

Open the lessons