CLOUD SERVICE · AWS

Amazon Security Lake - Amazon Security Lake centralizes security data from AWS environments, external providers, and custom sources in a customer-owned data lake on Amazon S3.

Amazon Security Lake collects supported AWS security logs and findings, converts them to a common schema, and organizes them for analysis across accounts and Regions. Engineers can connect analytics tools and security platforms through controlled subscriber access while retaining control over storage, retention, and encryption.

Cloud Services Hub →

What is Amazon Security Lake

Read the extensive description

Amazon Security Lake provides a managed foundation for collecting and organizing security telemetry in your AWS account. It stores data in Amazon S3 rather than providing a built-in investigation console or detection engine. Security teams use the resulting lake with query services, security information and event management platforms, and other analytics applications. This separates collection and storage from the tools used to investigate or detect threats.

For supported AWS sources, Security Lake handles collection and converts records into the Open Cybersecurity Schema Framework, or OCSF. It stores normalized records in Apache Parquet format, which supports efficient analytical queries. Supported sources include AWS CloudTrail events, Amazon VPC Flow Logs, Amazon Route 53 Resolver query logs, and AWS Security Hub findings. Enable only the sources and source versions needed for your investigations, and verify availability in each deployment Region.

Custom sources allow applications and external security products to contribute data. Unlike supported AWS sources, custom producers must supply data that meets Security Lake requirements, including the applicable OCSF schema and Parquet format. Treat schema validation, delivery permissions, and producer monitoring as part of the integration rather than assuming Security Lake will transform arbitrary incoming records.

In a multi-account environment, use AWS Organizations and a delegated administrator to manage collection centrally. Configure the accounts, Regions, and sources included in the lake. Regional data collection can be combined with rollup Regions to consolidate copies for analysis. Choose this topology deliberately because aggregation affects data residency, storage consumption, replication traffic, and the location in which analysts run queries.

Security Lake uses AWS Glue Data Catalog metadata and AWS Lake Formation permissions to support governed access. Subscribers can receive access to S3 objects or query access through Lake Formation, depending on their integration model. Data-access subscribers can receive notifications when new objects become available. Grant subscribers only the accounts, Regions, and source datasets they require, and ensure encryption key policies allow the intended access.

Configure retention and storage transitions according to investigative and compliance requirements. A useful starting point is to keep recent data readily queryable and move older data to lower-cost storage where appropriate. Account for the retrieval behavior of the selected storage class before relying on archived data during an incident.

Operational readiness requires more than enabling collection. Monitor source delivery, custom producer health, subscriber processing, and access failures. Validate representative queries against each enabled source and OCSF version, and test whether investigators can retrieve the required history. Budget for Security Lake processing as well as S3 storage, analytical queries, encryption operations, and any applicable replication or transfer charges.

Key Amazon Security Lake Features

Managed collection of AWS security telemetry

Enable supported AWS log and finding sources without building a separate collection pipeline for each one. Select sources by account and Region to match your investigation requirements and control ingestion volume.

OCSF normalization and Parquet storage

Security Lake converts supported AWS source records into OCSF and stores them in Apache Parquet format. Analysts can correlate common fields across different sources while using column-oriented query engines.

Customer-owned S3 storage

Security data resides in S3 buckets in your account. Configure retention, storage transitions, and encryption requirements while preserving the permissions needed by Security Lake.

Organization-wide administration

Use AWS Organizations and a delegated administrator to manage collection across member accounts. Centrally configure which accounts and sources participate instead of maintaining independent pipelines in every account.

Regional collection and rollup

Collect data in enabled Regions and configure rollup Regions for consolidated access. Evaluate replication costs and data residency requirements before selecting aggregation destinations.

Custom source ingestion

Integrate external security products and internal applications through custom sources. Producers must deliver records in the required OCSF-compatible schema and Parquet format.

Controlled subscriber access

Create subscribers for object-level data access or Lake Formation-based query access. Scope each subscriber to the source datasets it needs rather than granting broad bucket access.

Cataloged analytical datasets

Security Lake maintains dataset metadata through AWS Glue Data Catalog and integrates with Lake Formation permissions. Query-capable consumers can use this metadata to discover and analyze authorized tables.

Amazon Security Lake Use Cases

Cross-account incident investigation

Centralize relevant telemetry from production, development, and shared-services accounts so investigators can trace activity across account boundaries. Use a query engine to correlate API activity, network flows, and DNS requests.

Threat hunting across different data sources

Use OCSF-normalized fields to investigate suspicious identities, IP addresses, or resources across multiple datasets. Validate source-specific field mappings before building reusable hunting queries.

Supplying data to security analytics platforms

Give a SIEM or detection platform subscriber access to selected datasets. Object notifications can drive incremental ingestion without repeatedly scanning the entire lake.

Security telemetry retention

Retain security records independently of the operational retention settings in downstream analysis tools. Apply storage lifecycle policies that balance historical investigation needs with storage and retrieval costs.

Combining AWS and external security data

Publish endpoint, identity, or application security events through custom sources and analyze them alongside AWS telemetry. Build producer-side validation so malformed records do not undermine investigations.

Amazon Security Lake pricing models

Analytics and downstream consumption

Query engines and connected security platforms have their own pricing. For example, Athena queries incur Athena charges, while a subscriber platform may charge for ingestion, processing, or retained data.

Custom and third-party source data

Security Lake does not charge its ingestion and normalization fees for third-party or custom source data. Producers remain responsible for preparing compliant data, and storage, delivery, and other supporting services can still incur charges.

Storage and supporting AWS services

Amazon S3 storage, requests, lifecycle transitions, retrieval, replication, and applicable data transfer are billed separately. Include AWS KMS and other supporting service usage where applicable.

Usage-based AWS data ingestion and normalization

Security Lake charges for ingestion and normalization of supported AWS source data according to its pricing dimensions. Rates depend on the Region and source category; use source volumes and the applicable billing definitions when estimating costs.

Services Amazon Security Lake integrates with

A common architecture places Security Lake administration in a dedicated security account and uses AWS Organizations to manage collection across member accounts. Data is stored in Regional S3 buckets, with optional rollup Regions for consolidated analysis. AWS Glue Data Catalog describes the resulting datasets, while Lake Formation controls query-access subscribers. Encryption permissions must align with both the collection workflow and the consumers that need to read the data.For interactive investigation, analysts commonly use Athena against authorized Security Lake tables. Queries should restrict time ranges and relevant partitions, and select only necessary columns to limit scanned data. Security Hub findings provide useful investigation starting points that can be correlated with API activity, network flows, and DNS telemetry.For continuous downstream processing, configure a data-access subscriber and use SQS notifications to trigger retrieval of newly available S3 objects. A SIEM or custom analytics worker should tolerate retries and process objects idempotently. External producers can also contribute custom sources, but must prepare compliant OCSF and Parquet data before delivery. Across both ingestion and consumption paths, monitor delivery health, access failures, schema compatibility, and the cost of storage, queries, and replication.

Amazon Athena

Athena can query Security Lake datasets using catalog metadata and the required Lake Formation permissions. For example, investigators can search CloudTrail activity for a compromised role and correlate the results with network telemetry. Filter by relevant partitions and select only needed columns to reduce scanned data.

Open Amazon Athena →

AWS Glue

Security Lake uses AWS Glue Data Catalog to describe its security datasets. For example, an analyst can discover a source table and inspect its schema before writing a query. Validate table and schema versions when maintaining saved queries.

Open AWS Glue →

AWS Lake Formation

Lake Formation provides the permissions model for Security Lake query-access subscribers. For example, grant an analytics account access to selected security datasets without granting unrestricted access to all lake contents. Coordinate Lake Formation grants with IAM permissions and encryption access.

Open AWS Lake Formation →

AWS Organizations

AWS Organizations enables centralized Security Lake administration across member accounts through a delegated administrator. For example, a security account can manage collection for application accounts and configure organization-wide onboarding behavior. Align account inclusion and Regional settings with your organization's security boundaries.

Open AWS Organizations →

AWS Security Hub

Security Hub findings are a supported Security Lake source, allowing findings to be analyzed alongside underlying telemetry. For example, investigate a resource associated with a finding by querying its API activity and related network flows. Security Lake stores the data for analysis; it does not replace Security Hub's finding-management workflows.

Open AWS Security Hub →

Amazon S3

Security Lake stores normalized security records in customer-owned S3 buckets. For example, retain recent records in an immediately accessible storage class and transition older objects according to investigation and retention requirements. Account for retrieval constraints before moving data to archival storage.

Open Amazon S3 →