CLOUD CONCEPT

Data Mesh

Data Mesh is a decentralized data operating model in which business domains own data products while sharing a self-service platform and federated governance standards.

Cloud Concepts Hub →

What is Data Mesh

Data Mesh distributes responsibility for analytical data to the teams that understand its business meaning, rather than making a central data team responsible for every dataset. It combines domain ownership, data as a product, self-service infrastructure, and federated computational governance to support scalable, controlled data sharing.

Data Mesh is an organizational and architectural approach to managing analytical data across business domains. Instead of routing every ingestion request, transformation, and quality issue through one central data team, domains such as sales, payments, and logistics own and publish their data products. A shared platform supplies reusable capabilities, while enterprise-wide governance establishes rules that all domains must follow. Decentralizing ownership does not mean abandoning centralized security requirements.

A data product is more than a table or a collection of files. It has an accountable owner, documented business meaning, discoverable metadata, supported access interfaces, and measurable expectations for quality and freshness. For example, the payments domain might publish a daily settlements product with a documented schema, currency definitions, reconciliation checks, and a commitment to availability by a specified time. Consumers should understand how to request access, report defects, and prepare for breaking schema changes.

In the cloud, domains can operate in separate accounts, projects, or subscriptions while using shared provisioning templates and governance services. A platform team provides self-service capabilities for storage, pipelines, catalogs, identity integration, monitoring, and policy enforcement. Domain teams remain responsible for their products, but they should not need to build a separate security or deployment framework for each one. Physical separation is optional: multiple domains can also publish products from a shared lake or warehouse.

Federated computational governance combines enterprise-wide requirements with domain participation and automated enforcement. Common rules can cover sensitive-data classification, approved regions, encryption, retention, ownership metadata, and audit logging. Domain owners contribute business-specific definitions and access decisions within those boundaries. Policies should be enforced through deployment checks and runtime controls, not merely documented in a governance handbook.

Sharing patterns include governed access to producer-managed tables, warehouse data sharing, APIs, event streams, and controlled copies. Direct access can reduce duplication, but it requires compatible authorization and query mechanisms. Copies may support isolation or performance, but they introduce additional retention, deletion, and consistency obligations. Cross-account sharing still requires appropriate identity permissions, resource permissions, and, where applicable, encryption-key permissions. Catalog discovery or subscription approval alone does not necessarily authorize access to the underlying data.

Privacy requirements remain applicable across domain boundaries. A marketing team might receive a customer-segment product containing pseudonymous identifiers instead of names and email addresses. Access should be limited to approved purposes and necessary attributes, with row or column restrictions where supported. Pseudonymized data can remain personal data; it is not automatically anonymous. Retention and deletion processes must account for downstream copies, and access revocation does not erase data already exported by a consumer.

The trade-offs include more distributed operational responsibility, potential duplication, inconsistent definitions, and additional coordination for cross-domain joins. Shared identifiers, data contracts, quality checks, and clear ownership help manage these risks. Data Mesh is not a service, a service mesh, or simply a multi-account data lake. It can use lakes and warehouses, and a central platform team remains useful. In exam scenarios, look for domain-owned data products, reusable platform capabilities, and federated governance rather than assuming that decentralization means unrestricted sharing.

Key aspects of Data Mesh

Data Mesh combines domain accountability and product-quality data with shared platform capabilities, interoperable standards, and automated governance.

Domain-oriented ownership

Business-aligned teams own the lifecycle, meaning, quality, and support of their analytical data products. Assign an accountable owner so consumers know who handles defects and change requests.

Data as a product

Publish data with documentation, discoverable metadata, access interfaces, and explicit quality and freshness objectives. Manage schema evolution and deprecation as consumer-facing changes rather than internal implementation details.

Self-service data platform

Provide reusable infrastructure, deployment templates, observability, and access-management workflows. This reduces repeated engineering work without transferring product accountability away from domain teams.

Federated computational governance

Define shared policies collaboratively across governance, platform, security, and domain teams. Automate enforceable requirements such as classification metadata, approved storage locations, and access restrictions.

Governed sharing and privacy

Choose between direct access, APIs, streams, and controlled copies according to consumer needs and security constraints. Apply least privilege, data minimization, retention controls, and auditing across domain boundaries.

Cross-domain interoperability

Use agreed identifiers, metadata standards, and data contracts so independently maintained products can work together. Validate compatibility and communicate breaking changes before consumers are affected.

Cloud services for Data Mesh

Domain teams can store lake data in Amazon S3, maintain technical metadata and pipelines with AWS Glue, and expose warehouse products through Amazon Redshift. AWS Lake Formation governs supported lake access and cross-account sharing, while Amazon DataZone adds business discovery and subscription workflows. AWS CloudTrail supplies audit evidence for supported operations, supplemented by query and access logs. These services provide building blocks rather than a complete operating model: teams must still assign product owners, define quality objectives, establish privacy rules, and coordinate policy enforcement across storage, identity, encryption, and query layers.

Amazon DataZone

Provides business-oriented cataloging, discovery, publishing, and subscription workflows for data assets. It supports collaboration between producers and consumers, while underlying services and configured integrations enforce data access.

Open Amazon DataZone →
AWS Lake Formation image AWS Lake Formation

Supports centralized permission management and governed cross-account sharing for supported data lake resources. Fine-grained controls and Lake Formation tag-based access control can implement shared policies across domain-owned datasets.

Open AWS Lake Formation →
AWS Glue image AWS Glue

The AWS Glue Data Catalog stores technical metadata used by analytics engines and governance integrations. Glue processing and data quality capabilities help domain teams build pipelines and validate published products.

Open AWS Glue →
Amazon Simple Storage Service image Amazon S3

Provides object storage for domain-owned raw and curated datasets. Bucket policies, encryption, lifecycle configuration, and appropriate access boundaries support the storage layer, but do not establish a Data Mesh operating model by themselves.

Open Amazon S3 →
Amazon Redshift image Amazon Redshift

Provides warehouse-based data products and supported data sharing across workgroups, clusters, or accounts without manually copying shared data into each consumer warehouse. Producers must still define ownership, consumer permissions, and change-management expectations.

Open Amazon Redshift →
AWS CloudTrail image AWS CloudTrail

Records supported API activity for auditing governance and access-management operations across accounts. Configure relevant event coverage, including applicable data events, and supplement it with service-specific query and access logs where needed.

Open AWS CloudTrail →