CLOUD SERVICE · AWS

AWS Entity Resolution - AWS Entity Resolution is a managed service for matching and linking records that represent the same entity across datasets.

AWS Entity Resolution reconciles records across applications and data sources using configurable rules, machine learning, or supported external identity providers. It produces match identifiers and ID mappings that engineers can use for deduplication, cross-system analytics, and downstream data consolidation.

Cloud Services Hub →

What is AWS Entity Resolution

Read the extensive description

AWS Entity Resolution helps identify when different records refer to the same underlying entity. For example, a customer may appear in a CRM export, a billing database, and an online transaction dataset with different identifiers or contact details. The service compares selected attributes and assigns match identifiers to records determined to belong together. These identifiers provide a linkage layer for downstream processing without requiring applications to share a common source-system identifier.

A typical implementation starts with input data in Amazon S3 and table metadata in the AWS Glue Data Catalog. Engineers create a schema mapping that identifies the unique record identifier and maps source columns to supported attribute types, such as names, addresses, email addresses, and phone numbers. Schema mappings also specify which fields are passed through to output. Establish stable source identifiers and consistent field meanings before running workflows so that results remain traceable to their original records.

Matching workflows define how records are compared. Rule-based matching uses configurable, ordered rules that combine attributes, making it suitable for requirements such as matching on email address or on a combination of name and address. Machine learning-based matching identifies relationships that may be difficult to express with exact rules. Provider-based matching uses supported external identity providers and requires the appropriate provider access or subscription. The supported inputs, outputs, and configuration options differ by matching method.

Matching jobs process the configured input sources and write results to an S3 output location. Results contain match identifiers and method-specific information that downstream systems can use to interpret the links. These outputs are not automatically curated master records. Selecting an authoritative address, resolving conflicting names, and deciding whether to merge application records generally require separate business logic and, for ambiguous cases, manual review.

ID mapping workflows support a related but distinct operation: translating source identifiers into target identifiers using supported rule-based or provider-based methods. Depending on the configuration, this can involve datasets within an account or collaboration with another account. Design the required permissions and data-sharing arrangements before implementing cross-account workflows. Do not assume that a matching workflow and an ID mapping workflow have interchangeable configuration or output semantics.

Operational controls should include narrowly scoped IAM permissions, restricted S3 prefixes, and appropriate AWS KMS key policies. Validate regional availability, quotas, and matching-method constraints before committing to a design. Evaluate results against representative labeled records, measuring false matches and missed matches rather than relying only on job completion. Treat linkage outputs as sensitive derived data: a match does not establish consent or authorize a new use of personal information. Apply retention, access, and permitted-use policies throughout the pipeline.

Key AWS Entity Resolution Features

Schema mappings

Map source columns to entity attributes and designate a unique identifier for each input record. Explicit mappings establish consistent field meanings across datasets and control which fields are passed through to results.

Rule-based matching

Define ordered rules that compare combinations of attributes. Use this method when matching criteria must be predictable and explainable, such as linking records with the same email address.

Machine learning-based matching

Use managed machine learning to identify records representing the same entity despite variations in their attributes. Test the resulting links against labeled examples before using them for consequential decisions.

Provider-based matching

Use supported external identity providers to resolve records through their identity services. Configure the required provider subscription or access and confirm that the provider supports the intended attributes and use case.

Match identifiers and S3 outputs

Write matching results to Amazon S3 with identifiers that connect records in the same matched group. Use the outputs for analytical joins or as input to a separate master-data consolidation process.

ID mapping workflows

Create source-to-target identifier mappings using supported rule-based or provider-based configurations. This supports systems that need explicit ID translation rather than only matched groups.

Managed job execution

Start matching and ID mapping jobs through the console or APIs and inspect their status and processing metrics. Downstream pipelines can wait for successful completion before consuming results.

Access and encryption controls

Use IAM roles to authorize workflow operations and access to input and output resources. Configure supported AWS KMS encryption settings and test key policies alongside bucket and role permissions.

AWS Entity Resolution Use Cases

Customer record reconciliation

Link CRM, billing, and commerce records to create a shared customer linkage layer. Apply separate business rules to choose authoritative attributes and resolve conflicting values.

Dataset deduplication

Identify records that describe the same entity within or across datasets. Route uncertain matches to review rather than deleting or merging source records solely because they share a match identifier.

Cross-system analytics

Join datasets that lack a common application identifier using the matching outputs. Validate linkage quality before using the joined data for customer counts, attribution, or financial reporting.

Partner identifier translation

Map source identifiers to identifiers used by another dataset or supported identity provider. Establish data-sharing permissions and permitted uses before processing or distributing mappings.

Migration reconciliation

Compare legacy and replacement-system records to identify corresponding entities during a migration. Use unmatched records and questionable links to populate exception-handling queues.

AWS Entity Resolution pricing models

ID mapping workflows

Pricing depends on the ID mapping method. Rule-based ID mapping uses record-processing charges, while provider-based ID mapping follows the selected provider's pricing without an additional AWS Entity Resolution processing charge.

Provider-based matching

AWS Entity Resolution does not impose an additional processing charge for provider-based matching; the selected provider's fees apply. Review the provider's subscription, licensing, and usage terms when budgeting.

Rule-based and machine learning matching

AWS Entity Resolution charges for rule-based and machine learning-based matching according to the number of records processed and the applicable matching-method rate. Include processing frequency and reruns when estimating workload costs.

Supporting AWS services

Amazon S3 storage and requests, AWS KMS usage, data preparation, query execution, and orchestration can incur separate charges. Include these supporting services in the pipeline budget even when a matching method has no additional AWS Entity Resolution processing charge.

Services AWS Entity Resolution integrates with

A common architecture uses Amazon S3 as the data exchange layer and the AWS Glue Data Catalog to describe input datasets. Upstream ingestion or Glue ETL jobs prepare stable record identifiers and consistent attribute formats before an Entity Resolution workflow runs. Results are written to a separate S3 location, where downstream processing can apply survivorship rules or publish identity linkage tables.AWS Step Functions can coordinate preparation, job submission, status polling, and downstream processing through AWS SDK integrations. For example, a state machine can wait for a matching job to succeed before exposing its output to Athena queries. Preserve source identifiers and dataset context so consumers can trace links back to the originating systems.IAM and AWS KMS protect the pipeline's data and operations. Scope roles to required catalog tables, bucket prefixes, and encryption keys, and separate workflow administration from access to sensitive results. CloudTrail supplies an audit trail of API activity. Apply retention and permitted-use controls to matching outputs, and monitor both processing failures and the quality of the resulting links.

Amazon Athena

Athena can query matching results in S3 after appropriate table metadata is defined. For example, join matched customer records to transaction data to calculate activity across billing and commerce systems. Preserve source identifiers so analytical results retain lineage to the original records.

Open Amazon Athena →

AWS Glue

The AWS Glue Data Catalog provides table metadata for S3-backed input datasets. For example, define catalog tables for customer exports and reference them when configuring workflow inputs. Separate AWS Glue ETL jobs can standardize column types and remove malformed records before matching.

Open AWS Glue →

AWS Step Functions

Step Functions can orchestrate Entity Resolution API operations through AWS SDK integrations. For example, start a matching job after data preparation, poll its status with wait states, and launch downstream processing only after success. Include failure handling and grant the state machine role the necessary API permissions.

Open AWS Step Functions →

AWS CloudTrail

CloudTrail records Entity Resolution API activity for auditing. For example, inspect events to determine who changed a workflow configuration or started a job. API auditing complements operational status checks but does not measure matching accuracy.

Open AWS CloudTrail →

Amazon S3

Amazon S3 stores input files and workflow outputs. For example, place CRM and billing exports in separate input prefixes and write matching results to a restricted output prefix. Configure the workflow role, bucket policies, and encryption permissions for each location.

Open Amazon S3 →