AWS Certified Data Engineer Associate Interactive Course

Throughput and latency characteristics for AWS services that ingest data

Data ingestion patterns

Streaming data ingestion

Batch data ingestion

Replayability of data ingestion pipelines

Stateful and stateless data transactions

Reading data from streaming sources

Reading data from batch sources

Implementing appropriate configuration options for batch ingestion

Consuming data APIs

Setting up schedulers by using Amazon EventBridge, Apache Airflow, or time-based schedules for jobs and crawlers

Setting up event triggers

Calling a Lambda function from Amazon Kinesis

Creating allowlists for IP addresses to allow connections to data sources

Implementing throttling and overcoming rate limits

Managing fan-in and fan-out for streaming data distribution

Creation of ETL pipelines based on business requirements

Volume, velocity, and variety of data

Cloud computing and distributed computing

How to use Apache Spark to process data

Intermediate data staging locations

Connecting to different data sources

Integrating data from multiple sources

Optimizing costs while processing data

Optimizing container usage for performance needs

Implementing data transformation services based on requirements

Transforming data between formats

Troubleshooting and debugging common transformation failures and performance issues

Creating data APIs to make data available to other systems by using AWS services

How to integrate various AWS services to create ETL pipelines

Event-driven architecture

How to configure AWS services for data pipelines based on schedules or dependencies

Serverless workflows

Building data pipelines for performance, availability, scalability, resiliency, and fault tolerance

Implementing and maintaining serverless workflows

Using orchestration services to build workflows for data ETL pipelines

Using notification services to send alerts

Continuous integration and continuous delivery

SQL queries

Infrastructure as code for repeatable deployments

Distributed computing

Data structures and algorithms

SQL query optimization

Optimizing code to reduce runtime for data ingestion and transformation

Configuring Lambda functions to meet concurrency and performance needs

Performing SQL queries to transform data

Structuring SQL queries to meet data pipeline requirements

Using Git commands to perform actions such as creating, updating, cloning, and branching repositories

Using the AWS Serverless Application Model to package and deploy serverless data pipelines

Using and mounting storage volumes from within Lambda functions

Storage platforms and their characteristics

Storage services and configurations for specific performance demands

Data storage formats

How to align data storage with data migration requirements

How to determine the appropriate storage solution for specific access patterns

How to manage locks to prevent access to data

Configuring the appropriate storage services for specific access patterns and requirements

Applying storage services to appropriate use cases

Implementing the appropriate storage services for specific cost and performance requirements

Integrating migration tools into data processing systems

Implementing data migration or remote access methods

How to create a data catalog

Data classification based on requirements

Components of metadata and data catalogs

Using data catalogs to consume data from the data’s source

Building and referencing a data catalog

Discovering schemas and using AWS Glue crawlers to populate data catalogs

Synchronizing partitions with a data catalog

Creating new source or target connections for cataloging

Appropriate storage solutions to address hot and cold data requirements

How to optimize the cost of storage based on the data lifecycle

How to delete data to meet business and legal requirements

Data retention policies and archiving strategies

How to protect data with appropriate resiliency and availability

Performing load and unload operations to move data between Amazon S3 and Amazon Redshift

Managing S3 Lifecycle policies to change the storage tier of S3 data

Expiring data when it reaches a specific age by using S3 Lifecycle policies

Managing S3 versioning and DynamoDB TTL

Data modeling concepts

How to ensure accuracy and trustworthiness of data by using data lineage

Best practices for indexing, partitioning strategies, compression, and other data optimization techniques

How to model structured, semi-structured, and unstructured data

Schema evolution techniques

Designing schemas for Amazon Redshift, DynamoDB, and Lake Formation

Addressing changes to the characteristics of data

Performing schema conversion

Establishing data lineage by using AWS tools

How to maintain and troubleshoot data processing for repeatable business outcomes

API calls for data processing

Which services accept scripting

Orchestrating data pipelines

Troubleshooting Amazon managed workflows

Calling SDKs to access Amazon features from code

Using the features of AWS services to process data

Consuming and maintaining data APIs

Preparing data transformation

Querying data

Using Lambda to automate data processing

Managing events and schedulers

Tradeoffs between provisioned services and serverless services

SQL queries

How to visualize data for analysis

When and how to apply cleansing techniques

Data aggregation, rolling average, grouping, and pivoting

Visualizing data by using AWS services and tools

Verifying and cleaning data

Using Athena to query data or to create views

Using Athena notebooks that use Apache Spark to explore data

How to log application data

Best practices for performance tuning

How to log access to AWS services

Amazon Macie, AWS CloudTrail, and Amazon CloudWatch

Extracting logs for audits

Deploying logging and monitoring solutions to facilitate auditing and traceability

Using notifications during monitoring to send alerts

Troubleshooting performance issues

Using CloudTrail to track API calls

Troubleshooting and maintaining pipelines

Using Amazon CloudWatch Logs to log application data

Analyzing logs with AWS services

Data sampling techniques

How to implement data skew mechanisms

Data validation

Data profiling

Running data quality checks while processing the data

Defining data quality rules

Investigating data consistency

VPC security networking concepts

Differences between managed services and unmanaged services

Authentication methods

Differences between AWS managed policies and customer managed policies

Updating VPC security groups

Creating and updating IAM groups, roles, endpoints, and services

Creating and rotating credentials for password management

Setting up IAM roles for access

Applying IAM policies to roles, endpoints, and services

Authorization methods

Principle of least privilege as it applies to AWS security

Role-based access control and expected access patterns

Methods to protect data from unauthorized access across services

Creating custom IAM policies when a managed policy does not meet the needs

Storing application and database credentials

Providing database users, groups, and roles access and authority in a database

Managing permissions through Lake Formation

Data encryption options available in AWS analytics services

Differences between client-side encryption and server-side encryption

Protection of sensitive data

Data anonymization, masking, and key salting

Applying data masking and anonymization according to compliance laws or company policies

Using encryption keys to encrypt or decrypt data

Configuring encryption across AWS account boundaries

Enabling encryption in transit for data

How to log application data

How to log access to AWS services

Centralized AWS logs

Using CloudTrail to track API calls

Using CloudWatch Logs to store application logs

Using AWS CloudTrail Lake for centralized logging queries

Analyzing logs by using AWS services

Integrating various AWS services to perform logging

How to protect personally identifiable information

Data sovereignty

Granting permissions for data sharing

Implementing PII identification

Implementing data privacy strategies to prevent backups or replications of data to disallowed AWS Regions

Managing configuration changes that have occurred in an account