# Data Quality

Data quality is the degree to which data is accurate, complete, consistent, valid, timely, and suitable for its intended use.

Cloud concept. Updated 2026-10-29. Page: https://blowstack.com/tools/cloud-concepts-hub/data-quality

> Data quality determines whether engineers can rely on data for analytics, application behavior, and AI training or inference. In secure data engineering, quality assessments work alongside privacy protections, access controls, and integrity checks to reduce unreliable outputs, unauthorized changes, and inappropriate use of sensitive information.

Practice exams covering Data Quality: [AIB-C01 (beta) AWS Certified AI Business Strategist](https://blowstack.com/practice-exams/aws-certified-ai-business-strategist), [AIF-C01 AWS Certified AI Practitioner](https://blowstack.com/practice-exams/aws-certified-ai-practitioner), [DEA-C01 AWS Certified Data Engineer - Associate](https://blowstack.com/practice-exams/aws-certified-data-engineer-associate), [AIP-C01 AWS Certified Generative AI Developer - Professional](https://blowstack.com/practice-exams/aws-certified-generative-ai-developer-professional), [MLA-C02 (beta) AWS Certified Machine Learning Engineer - Associate](https://blowstack.com/practice-exams/aws-certified-machine-learning-engineer-associate).

## 6 key aspects of Data Quality

Reliable data requires measurable, purpose-specific checks backed by continuous monitoring, traceability, secure access, and privacy-aware processing.

1. **Purpose-specific acceptance criteria**: Define quality expectations around the consuming workload and its risks. A reporting dataset and a real-time decision system may need different freshness limits and error tolerances.
2. **Measurable quality dimensions**: Measure accuracy, completeness, consistency, validity, uniqueness, and timeliness separately. Passing a schema check does not establish that values are factually correct.
3. **Continuous validation and remediation**: Run automated checks at ingestion and transformation boundaries, then monitor trends. Assign owners and define when failures should block publication, quarantine records, or trigger investigation.
4. **Provenance and protected integrity**: Track sources, dataset versions, and transformations while restricting and auditing write access. Use integrity mechanisms to detect unauthorized or accidental changes without confusing unchanged data with accurate data.
5. **Privacy-aware processing**: Minimize sensitive data and apply appropriate masking, tokenization, or privacy techniques. Reassess quality after these transformations because they can change analytical utility and join behavior.
6. **AI dataset suitability**: Assess label reliability, representative coverage, and training-evaluation separation. Monitor production input changes and investigate their effect on model performance rather than assuming every change is harmful.

## Cloud services for Data Quality

A practical workflow stores source data in Amazon S3 and uses AWS Glue to transform it and evaluate quality rules before publishing curated datasets. Failed records can be isolated for investigation, while versions and validation results support reproducibility. AWS Lake Formation controls supported access to curated lake data, and Amazon Macie helps identify sensitive information requiring additional protection. For deployed AI workloads, Amazon SageMaker AI provides monitoring capabilities that can flag input deviations from established baselines. Together, these services support validation, controlled access, privacy discovery, and ongoing monitoring, but engineers must still define acceptance criteria, investigate exceptions, and verify business correctness.

- **[AWS Glue](https://blowstack.com/tools/cloud-services-hub/aws-glue)**: AWS Glue Data Quality supports defining and evaluating quality rules for cataloged data and data processed in Glue ETL jobs. Engineers can use rule results to identify completeness, uniqueness, and other quality failures and configure pipeline responses.
- **[Amazon S3](https://blowstack.com/tools/cloud-services-hub/amazon-s3)**: Amazon S3 can store source, quarantined, and curated datasets with separate access policies. Object versioning and checksums support recovery and integrity verification, but additional validation is required to determine whether stored records are correct.
- **[AWS Lake Formation](https://blowstack.com/tools/cloud-services-hub/aws-lake-formation)**: AWS Lake Formation provides fine-grained permissions for supported access to data lake tables and columns, including row- and cell-level filtering where supported. It helps restrict sensitive data exposure and access to curated datasets, while quality rules remain a separate responsibility.
- **[Amazon Macie](https://blowstack.com/tools/cloud-services-hub/amazon-macie)**: Amazon Macie discovers sensitive data in Amazon S3, helping engineers identify information that needs restricted access or privacy-preserving transformation. It does not cleanse records, anonymize datasets, or certify their quality.

## Data Quality exam questions

Real practice exam questions on Data Quality, with the correct answer and its explanation.

### Question 1 (AIB-C01 (beta))

**Question:**

A commercial lender is preparing an AI model to assess loan applications. Historical testing shows excellent accuracy, but a review finds that the training pipeline attaches each applicant's most recently revised financial statement to every past application. Some revisions were published only after the historical lending decision. All statements are accurate, and schema and completeness checks pass. Leadership requires credible evidence of model reliability before deployment. Which approach should the AI business strategist recommend?

**Correct answer:** Use AWS Glue transformations to attach only statement versions available at each historical decision time, retain the relevant version and publication timestamps, and repeat training and evaluation on the corrected data.

**Explanation:** Point-in-time preparation prevents future information from entering historical examples. Retaining statement versions and publication timestamps makes the preparation auditable, and repeating training and evaluation provides more credible evidence of reliability.

### Question 2 (MLA-C02 (beta))

**Question:**

A retailer uses an AWS Glue ETL job to aggregate settled transactions into store-and-product daily sales features for ML training. Each output row contains a total_sales value. An authoritative ledger contains one row per transaction with a sale_amount value. Both datasets cover the same business day, use the same currency and fixed-precision decimal representation, and contain only positive sales amounts.

A recent transformation defect omitted some transactions while producing output with valid identifiers, no nulls, and plausible sales values. The engineer must add a managed data quality check that detects whether the output's total sales differ from the ledger's total sales. The number of output rows is expected to differ from the number of ledger rows.

Which check should the engineer implement?

**Correct answer:** Use an AWS Glue Data Quality AggregateMatch rule to compare SUM(total_sales) in the output with SUM(sale_amount) in the ledger and require equality.

**Explanation:** AggregateMatch compares aggregate query results across datasets. Comparing these sums directly verifies that the aggregation preserved the authoritative sales total, even though the datasets have different granularities and row counts.

### Question 3 (AIP-C01)

**Question:**

A distributor maintains a vector store for a generative AI assistant that answers questions about chemical handling. A custom AWS Step Functions workflow uses AWS Lambda functions to transform approved product records into passages and update their embeddings. Each source record includes a revision, a maximum storage temperature, and its temperature unit.

A transformation defect sometimes treats Fahrenheit values as Celsius. The resulting passages are complete, contain valid numbers, and generate embeddings successfully, but state unsafe storage limits. The authoritative source values are correct. Updates are processed serially for each product, and retrieval can exclude individual products through an existing eligibility registry.

Valid updates must become searchable within 5 minutes without waiting for unrelated products. After an update finishes processing, a product must not remain searchable with either an incorrect new limit or a superseded limit. Failed records must remain available for diagnosis and reprocessing.

Which maintenance design meets these requirements?

**Correct answer:** Add a Lambda validation step that independently converts the authoritative temperature into a canonical unit and compares it with the transformed passage's value and unit. Publish a revision only if they agree. On failure, mark the product ineligible for retrieval, quarantine the failed revision with its source and transformation details, and notify operators. Continue processing other products.

**Explanation:** This checks semantic accuracy against the authoritative source rather than merely checking format or processing success. Per-product validation allows valid updates to proceed independently. Excluding a failed product prevents both incorrect replacement content and superseded content from being served, while quarantine preserves the information needed for diagnosis and reprocessing.

### Question 4 (AIF-C01)

**Question:**

A manufacturer uses an AI application to recommend immediate adjustments to factory equipment based on sensor readings. During network interruptions, sensors buffer readings and upload them after connectivity returns. The uploaded files contain all required fields, and their checksums match the originals. However, the application sometimes recommends adjustments based on conditions that no longer exist. Which change to data preparation would best address this problem?

**Correct answer:** Compare each reading's measurement timestamp with an application-defined freshness limit and withhold outdated readings from immediate recommendations.

**Explanation:** Timeliness is essential when data supports immediate operational decisions. Checking when measurements were taken prevents delayed readings from being treated as current conditions, even when the files are complete and unchanged.

### Question 5 (DEA-C01)

**Question:**

A company is replacing a billing export process. An AWS Glue ETL job reads the legacy export and the replacement export from Amazon S3 as separate datasets. Both datasets contain invoice_id, account_id, and invoice_amount. All values are non-null, and invoice_id is unique in each dataset. Engineers have already verified that both datasets contain exactly the same invoice IDs.

The replacement export must associate exactly the same account_id and invoice_amount with each invoice_id as the legacy export. Matching schemas, row counts, and total invoice amounts are insufficient because values might have been assigned to the wrong invoices. The engineer must validate the exports by using built-in AWS Glue Data Quality rules without writing custom comparison code.

Which solution meets these requirements?

**Correct answer:** Configure DatasetMatch with the legacy export as the reference dataset, invoice_id as the matching key, and account_id and invoice_amount as the comparison columns. Require a match ratio of 1.0.

**Explanation:** DatasetMatch compares values across datasets using specified matching keys and comparison columns. With identical invoice ID sets already verified, requiring a match ratio of 1.0 establishes that every invoice has the same account and amount in both exports.

## Practice Data Quality in a full exam

Questions on Data Quality in each practice exam, with every answer option and an explanation for each:

- [AIB-C01 (beta)](https://blowstack.com/practice-exams/aws-certified-ai-business-strategist): 37 questions
- [DEA-C01](https://blowstack.com/practice-exams/aws-certified-data-engineer-associate): 13 questions
- [MLA-C02 (beta)](https://blowstack.com/practice-exams/aws-certified-machine-learning-engineer-associate): 7 questions
- [AIP-C01](https://blowstack.com/practice-exams/aws-certified-generative-ai-developer-professional): 4 questions
- [AIF-C01](https://blowstack.com/practice-exams/aws-certified-ai-practitioner): 1 question

## What is Data Quality

Data quality is fitness for a specific purpose, not a universal property of a dataset. A daily inventory snapshot might be adequate for demand forecasting but unsuitable for real-time order fulfillment. Engineers should define measurable acceptance criteria based on the consuming application, including the consequences of incorrect, missing, duplicated, or outdated records.

Common quality dimensions include accuracy, completeness, consistency, validity, uniqueness, and timeliness. Accuracy concerns whether a value reflects reality; validity concerns whether it follows an expected format or rule. A correctly formatted postal code can still be inaccurate for a customer's address. Similarly, a dataset with no missing values may contain fabricated defaults, duplicate customers, or stale measurements.

In cloud pipelines, assess quality during ingestion, after transformations, and before publishing data to consumers. Profile incoming datasets to understand distributions, null rates, schemas, and unexpected categories. Convert important expectations into automated checks, such as requiring transaction identifiers to be unique, rejecting impossible timestamps, and reconciling aggregate payment totals against an authoritative source. Track results over time rather than treating a successful initial assessment as permanent approval.

Use explicit failure handling. A pipeline might block publication when a required identifier is missing, quarantine malformed records for investigation, and issue a warning for a small increase in optional-field nulls. Preserve source data under appropriate access and retention controls so that corrections can be reproduced. Record dataset versions, transformation logic, ownership, and lineage to explain how a published value was produced.

AI workloads introduce additional quality concerns. Evaluate label accuracy, coverage of relevant populations and operating conditions, and separation between training and evaluation datasets. Duplicate or overlapping examples can inflate evaluation scores. A fraud dataset dominated by legitimate transactions can appear complete while providing too few useful fraud examples. Monitor production inputs for schema violations and distribution changes, but investigate changes before assuming they indicate corruption or degraded model performance.

Secure data engineering protects the quality process itself. Apply least-privilege access so ingestion systems, transformation jobs, and model developers receive only the permissions they need. Restrict writes to approved datasets, audit modifications, and validate source provenance to reduce accidental changes and data-poisoning risks. Checksums help detect unintended byte changes, while authenticated signatures can help establish origin; neither proves that the underlying facts or labels are correct. Encryption protects confidentiality but does not repair inaccurate data.

Privacy-enhancing techniques must also preserve fitness for use. Minimize collected attributes, tokenize identifiers where appropriate, and use masking or differential privacy when suitable for the use case. Tokenized data is not automatically anonymous, particularly when re-identification mappings remain available. Assess whether privacy transformations remove important predictive signals, disrupt joins, or disproportionately affect small groups.

Quality controls involve trade-offs among cost, latency, coverage, and risk. Full-dataset checks provide more coverage than sampling but can cost more and delay processing. Strict rejection rules may prevent bad data from reaching a model while also excluding legitimate unusual cases. Set risk-based thresholds with data owners, monitor exceptions, and remediate root causes rather than silently dropping every anomaly. More data is not necessarily better data, and passing automated rules does not establish fairness, security, or regulatory compliance.

## Learn Data Quality in lessons

12 short lessons on Data Quality in 5 certification courses.

- AIB-C01 (beta): Data Quality: fitness for AI use (Premium), https://app.blowstack.com/lessons/data-quality-fitness-for-ai-use-aib-c01
- MLA-C02 (beta): Data Quality: rule-based dataset validation (Premium), https://app.blowstack.com/lessons/data-quality-rule-based-dataset-validation-mla-c02
- AIP-C01: Data Quality: continuous vector-store validation (Premium), https://app.blowstack.com/lessons/data-quality-continuous-vector-store-validation-aip-c01
- AIF-C01: Data Integrity and Data Quality: preserved versus fit for use (Premium), https://app.blowstack.com/lessons/data-integrity-and-data-quality-preserved-versus-fit-for-use-aif-c01
- DEA-C01: Data Profiling vs Data Quality (Premium), https://app.blowstack.com/lessons/data-profiling-vs-data-quality-dea-c01
- AIB-C01 (beta): Data Quality: ongoing quality controls (Premium), https://app.blowstack.com/lessons/data-quality-ongoing-quality-controls-aib-c01
