# Amazon Data Firehose

Cloud service · AWS. Updated 2026-11-12. Page: https://blowstack.com/tools/cloud-services-hub/amazon-kinesis-data-firehose

> Amazon Data Firehose is a fully managed service designed to load streaming data in real time into data lakes, data stores, and analytics services such as Amazon S3, Amazon Redshift, Amazon OpenSearch Service, and Splunk.

Practice exams covering Amazon Data Firehose: [ANS-C01 AWS Certified Advanced Networking - Specialty](https://blowstack.com/practice-exams/aws-certified-advanced-networking-specialty), [CLF-C02 AWS Certified Cloud Practitioner](https://blowstack.com/practice-exams/aws-certified-cloud-practitioner), [DEA-C01 AWS Certified Data Engineer - Associate](https://blowstack.com/practice-exams/aws-certified-data-engineer-associate), [DVA-C02 AWS Certified Developer - Associate](https://blowstack.com/practice-exams/aws-certified-developer-associate), [DOP-C02 AWS Certified DevOps Engineer - Professional](https://blowstack.com/practice-exams/aws-certified-devops-engineer-professional), [MLA-C02 (beta) AWS Certified Machine Learning Engineer - Associate](https://blowstack.com/practice-exams/aws-certified-machine-learning-engineer-associate), [SCS-C03 AWS Certified Security - Specialty](https://blowstack.com/practice-exams/aws-certified-security-specialty), [SAA-C03 AWS Certified Solutions Architect - Associate](https://blowstack.com/practice-exams/aws-certified-solutions-architect-associate), [SAP-C03 AWS Certified Solutions Architect - Professional](https://blowstack.com/practice-exams/aws-certified-solutions-architect-professional).

## 9 key features of Amazon Data Firehose

Amazon Kinesis Data Firehose offers an easy-to-use, automatically scalable, near real-time data streaming service with capabilities for data transformation and extensive support for various analytics destinations, along with simplified monitoring and automation.

1. **Easy to Use**: Amazon Data Firehose provides a straightforward way to reliably load streaming data into data lakes, data stores, and analytics services. It eliminates the need to write applications or manage resources, facilitating quick data transfer.
2. **Automatic Scaling**: It automatically scales to match the throughput of your data and requires no ongoing administration. This ensures that your data streaming does not suffer from bottlenecks or require manual intervention for scaling.
3. **Near Real-time**: Amazon Kinesis Data Firehose enables near real-time analytics with minimal latency, typically less than 60 seconds, allowing businesses to quickly analyze and respond to incoming streaming data.
4. **Data Transformation**: It offers the ability to transform data before loading it into the destination, making it possible to convert raw streaming data into formats required by downstream analytics tools without additional processing layers.
5. **Supported Destinations**: Amazon Data Firehose supports a wide range of destinations including Amazon S3, Amazon Redshift, Amazon OpenSearch Service, and Splunk, making it versatile for various use cases and analytics needs.
6. **Monitoring and Automation**: With Amazon Kinesis Data Firehose, monitoring is simplified through integration with Amazon CloudWatch, and data streaming processes can be automated using AWS Lambda for event-driven processing tasks.
7. **Iceberg, Serverless, and HTTP Destinations**: Firehose also supports delivery to Apache Iceberg tables, Amazon OpenSearch Serverless, and custom HTTP endpoints, including endpoints owned by supported third-party providers.
8. **Kinesis Data Streams Source**: A Firehose stream can automatically read records from an existing Kinesis data stream and deliver them to its destination.
9. **Amazon MSK Source**: Firehose supports Amazon MSK as a source, with a default read-throughput quota of 10 MB per second per partition and a maximum record size of 10 MB when Lambda transformation is disabled. With Lambda enabled, the maximum record size is 6 MB; oversized records are delivered to an S3 error bucket.

## Where Amazon Data Firehose is used

Amazon Kinesis Data Firehose is used for real-time data analytics, log and event data capture, streamlining IoT data, and enabling machine learning model inference by capturing, transforming, and loading streaming data into AWS data stores.

1. **Real-time Data Analytics**: Amazon Data Firehose allows organizations to capture, transform, and load streaming data into AWS data stores, such as Amazon S3, Amazon Redshift, and Amazon OpenSearch Service, in real-time. This enables immediate analytics and insights using tools like Amazon Athena and Amazon QuickSight for data-driven decision-making.
2. **Log and Event Data Capture**: Organizations can use Amazon Data Firehose to efficiently collect, aggregate, and process log and event data from various sources, such as application logs, website clickstreams, and IoT devices. This facilitates centralized logging, monitoring, and analysis for operational excellence and compliance auditing.
3. **Streamlining IoT Data**: With Amazon Data Firehose, businesses can manage the vast influx of data from IoT devices and sensors by ingesting, transforming, and delivering this data to the appropriate analytics tools and storage solutions on AWS. This supports real-time and batch analytics, enabling actionable insights into IoT operations and customer behaviors.
4. **Machine Learning Model Inference**: Kinesis Data Firehose can be used to preprocess and deliver streaming data to Amazon SageMaker for real-time machine learning model inference. This integration allows businesses to add AI and ML capabilities to their applications, providing features like predictive analytics, anomaly detection, and personalized content delivery.

## How Amazon Data Firehose is priced

Amazon Kinesis Data Firehose pricing is based on the volume of data ingested, the use of data conversion features, and data transfer costs outside the region or to the internet.

- **Data Ingestion Pricing**: With Amazon Data Firehose, you pay for the amount of data you transmit through the service. The price is calculated based on the volume of data ingested into the service, measured in gigabytes.
- **Data Conversion Pricing**: If you enable built-in data conversion features, which convert JSON into Apache Parquet or Apache ORC, additional charges are applied. These charges are based on the volume of data processed by the conversion feature.
- **Data Transfer Pricing**: Data transferred out of Amazon Data Firehose to another AWS service (within the same region) is free. However, if the data is transferred to AWS services in different regions or to the internet, standard AWS data transfer fees apply.
- **Dynamic Partitioning Pricing**: When dynamic partitioning is enabled for Amazon S3 delivery, additional charges apply per gigabyte processed and per object delivered to S3; using jq for parsing also incurs processing-hour charges.
- **CloudWatch Logs Decompression Pricing**: Enabling decompression of CloudWatch Logs incurs an additional charge based on the volume of decompressed data.

## Services Amazon Data Firehose works with

- **[Amazon S3](https://blowstack.com/tools/cloud-services-hub/amazon-s3)**: Amazon Data Firehose can deliver streaming data to Amazon S3 buckets, providing durable storage with high availability for big data analytics.
- **[Amazon Redshift](https://blowstack.com/tools/cloud-services-hub/amazon-redshift)**: Amazon Data Firehose can load streaming data into Amazon Redshift tables by first delivering it to Amazon S3 and then issuing a Redshift COPY command, enabling near real-time analytics and querying of structured data.
- **[Amazon API Gateway](https://blowstack.com/tools/cloud-services-hub/amazon-api-gateway)**: Amazon Data Firehose can receive API Gateway REST API access logs for delivery to analytics and storage destinations. API Gateway can also use an AWS service integration to submit request data to a Firehose stream.
- **[Amazon Aurora](https://blowstack.com/tools/cloud-services-hub/amazon-aurora)**: Amazon Data Firehose can capture changes from supported Aurora MySQL and PostgreSQL databases and deliver them to Apache Iceberg tables. This supports continuously updated analytical copies of operational data.
- **[Amazon Aurora Serverless](https://blowstack.com/tools/cloud-services-hub/amazon-aurora-serverless)**: Amazon Data Firehose can capture database changes from supported Aurora Serverless v2 configurations and deliver them to Apache Iceberg tables. The database must have the required replication settings and permissions enabled.
- **[Amazon CloudFront](https://blowstack.com/tools/cloud-services-hub/amazon-cloudfront)**: Amazon Data Firehose can receive CloudFront standard access logs through CloudFront standard logging v2. Firehose buffers and delivers the logs to its configured destination.
- **[Amazon CloudWatch](https://blowstack.com/tools/cloud-services-hub/amazon-cloudwatch)**: Amazon Data Firehose receives CloudWatch Logs subscription data and CloudWatch metric streams for delivery to storage and monitoring destinations. Firehose also publishes its own delivery metrics and can write delivery-error logs to CloudWatch.
- **[Amazon Connect](https://blowstack.com/tools/cloud-services-hub/amazon-connect)**: Amazon Data Firehose can receive contact records through Amazon Connect's data-streaming configuration. Firehose delivers these records to a destination for contact-center reporting and analysis.
- **[Amazon EC2](https://blowstack.com/tools/cloud-services-hub/amazon-ec2)**: Amazon Data Firehose can receive files and logs collected by the Amazon Kinesis Agent running on EC2 instances. The agent monitors configured files and batches new records into a Firehose stream.
- **[Amazon ECS Anywhere](https://blowstack.com/tools/cloud-services-hub/amazon-ecs-anywhere)**: Amazon Data Firehose can receive container logs from ECS Anywhere tasks through FireLens log routing. This lets applications on external ECS instances use the same Firehose delivery pipeline as containers running in AWS.
- **[Amazon Elastic Container Service (Amazon ECS)](https://blowstack.com/tools/cloud-services-hub/amazon-elastic-container-service-(amazon-ecs))**: Amazon Data Firehose can receive ECS container logs through FireLens and its Fluent Bit Firehose output. A task's logging configuration selects the Firehose stream that receives its application logs.
- **[Amazon Elastic Kubernetes Service (Amazon EKS)](https://blowstack.com/tools/cloud-services-hub/amazon-elastic-kubernetes-service-(amazon-eks))**: Amazon Data Firehose can receive EKS container logs through the Firehose output in AWS for Fluent Bit. A logging agent can collect logs across cluster nodes and forward them into a centralized delivery stream.
- **[Amazon EventBridge](https://blowstack.com/tools/cloud-services-hub/amazon-eventbridge)**: Amazon Data Firehose can be a target for EventBridge rules and EventBridge Pipes. EventBridge filters and optionally transforms events before submitting them to the selected Firehose stream.
- **[Amazon Kinesis Data Streams](https://blowstack.com/tools/cloud-services-hub/amazon-kinesis-data-streams)**: Amazon Data Firehose can use a Kinesis data stream as its source, reading records automatically and delivering them to a configured destination. Firehose can transform or convert the records before delivery.
- **[Amazon Managed Service for Apache Flink](https://blowstack.com/tools/cloud-services-hub/amazon-managed-service-for-apache-flink)**: Amazon Data Firehose can receive output from Managed Service for Apache Flink applications through the Amazon Data Firehose sink connector. Flink can aggregate or enrich events before sending the results to Firehose for delivery.
- **[Amazon Managed Streaming for Apache Kafka](https://blowstack.com/tools/cloud-services-hub/amazon-managed-streaming-for-apache-kafka)**: Amazon Data Firehose can read records directly from a topic in a supported Amazon MSK cluster. Firehose manages consumption and delivery without requiring a separate Kafka consumer application.
- **[Amazon Managed Workflows for Apache Airflow (Amazon MWAA)](https://blowstack.com/tools/cloud-services-hub/amazon-managed-workflows-for-apache-airflow-(amazon-mwaa))**: Amazon Data Firehose can be used in MWAA workflows through the Amazon provider's Firehose operators. An Airflow task can submit batches of records to a Firehose stream as part of a scheduled data pipeline.
- **[Amazon OpenSearch Service](https://blowstack.com/tools/cloud-services-hub/amazon-opensearch-service)**: Amazon Data Firehose delivers streaming records to OpenSearch Service domains and OpenSearch Serverless collections for indexing and analysis. It can transform records before indexing and back up failed deliveries to S3.
- **[Amazon RDS](https://blowstack.com/tools/cloud-services-hub/amazon-rds)**: Amazon Data Firehose can capture changes from supported RDS for MySQL and RDS for PostgreSQL databases and deliver them to Apache Iceberg tables. This maintains an analytical copy of database data without a separately managed change-data-capture application.
- **[Amazon Route 53](https://blowstack.com/tools/cloud-services-hub/amazon-route-53)**: Amazon Data Firehose can receive Route 53 Resolver query logs through a Resolver query-logging configuration. Firehose delivers the DNS query records to a destination for security analysis or retention.
- **[Amazon Simple Email Service (Amazon SES)](https://blowstack.com/tools/cloud-services-hub/amazon-simple-email-service-(amazon-ses))**: Amazon Data Firehose can receive SES email-sending events through a configuration set's Firehose event destination. These records can include deliveries, bounces, complaints, opens, and clicks.
- **[Amazon Simple Notification Service (Amazon SNS)](https://blowstack.com/tools/cloud-services-hub/amazon-simple-notification-service-(amazon-sns))**: Amazon Data Firehose can subscribe to an SNS topic using the Firehose subscription protocol. SNS forwards matching messages to the stream using an IAM role authorized to write to Firehose.
- **[Amazon VPC](https://blowstack.com/tools/cloud-services-hub/amazon-vpc)**: Amazon Data Firehose can receive VPC Flow Logs directly for delivery to storage and analytics destinations. Firehose also supports VPC connectivity for supported private sources and destinations.
- **[AWS CloudFormation](https://blowstack.com/tools/cloud-services-hub/aws-cloudformation)**: Amazon Data Firehose delivery streams can be provisioned and updated with the AWS::KinesisFirehose::DeliveryStream CloudFormation resource. Templates can define sources, destinations, buffering, transformations, and encryption settings.
- **[AWS CloudTrail](https://blowstack.com/tools/cloud-services-hub/aws-cloudtrail)**: Amazon Data Firehose integrates with CloudTrail to record supported API activity against delivery streams. CloudTrail can audit stream-management operations and, when configured, record supported data events such as record ingestion.
- **[AWS Config](https://blowstack.com/tools/cloud-services-hub/aws-config)**: Amazon Data Firehose delivery streams are supported AWS Config resources. Config can record stream configuration changes and evaluate them against compliance rules.
- **[AWS Fargate](https://blowstack.com/tools/cloud-services-hub/aws-fargate)**: Amazon Data Firehose can receive logs from Fargate workloads through ECS FireLens or the built-in Fluent Bit logging configuration for EKS on Fargate. These integrations route container logs without requiring access to the underlying host.
- **[AWS Glue](https://blowstack.com/tools/cloud-services-hub/aws-glue)**: Amazon Data Firehose uses schemas in the AWS Glue Data Catalog to convert incoming JSON records to Parquet or ORC. Firehose also uses Glue catalog metadata when delivering records to Apache Iceberg tables.
- **[AWS Identity and Access Management (IAM)](https://blowstack.com/tools/cloud-services-hub/aws-identity-and-access-management-(iam))**: Amazon Data Firehose uses IAM policies to control who can create, configure, and write to delivery streams. Firehose assumes IAM service roles to read sources, invoke transformations, and write to destinations.
- **[AWS IoT Core](https://blowstack.com/tools/cloud-services-hub/aws-iot-core)**: Amazon Data Firehose can receive messages through an AWS IoT Core rule's Firehose action. The rule can filter or reshape MQTT message data before forwarding it to a delivery stream.
- **[AWS IoT Greengrass](https://blowstack.com/tools/cloud-services-hub/aws-iot-greengrass)**: Amazon Data Firehose can receive edge-device data through the Kinesis Firehose connector for AWS IoT Greengrass V1. The connector batches messages and sends them to a configured Firehose stream.
- **[AWS Key Management Service (AWS KMS)](https://blowstack.com/tools/cloud-services-hub/aws-key-management-service-(aws-kms))**: Amazon Data Firehose uses AWS KMS keys for server-side encryption of data held in delivery streams. IAM and KMS key policies control whether Firehose can use a customer-managed key.
- **[AWS Lake Formation](https://blowstack.com/tools/cloud-services-hub/aws-lake-formation)**: Amazon Data Firehose can deliver records to Apache Iceberg tables whose access is governed by Lake Formation. The Firehose delivery role must receive the required permissions for the destination tables and underlying data.
- **[AWS Lambda](https://blowstack.com/tools/cloud-services-hub/aws-lambda)**: Amazon Data Firehose can invoke a Lambda function to transform buffered records before delivery. The function can enrich, filter, or reformat records and return processing results to Firehose.
- **[AWS Network Firewall](https://blowstack.com/tools/cloud-services-hub/aws-network-firewall)**: Amazon Data Firehose can receive supported Network Firewall logs through the firewall's logging configuration. Firehose then delivers the records to a configured destination for investigation or retention.
- **[AWS PrivateLink](https://blowstack.com/tools/cloud-services-hub/aws-privatelink)**: Amazon Data Firehose supports interface VPC endpoints powered by AWS PrivateLink. Producers can call Firehose APIs over private network connections without using an internet gateway or NAT gateway.
- **[AWS Resource Groups and Tag Editor](https://blowstack.com/tools/cloud-services-hub/aws-resource-groups-and-tag-editor)**: Amazon Data Firehose delivery streams support tag-based discovery and organization through Resource Groups and Tag Editor. Teams can locate and group streams using tags such as application, environment, or owner.
- **[AWS Secrets Manager](https://blowstack.com/tools/cloud-services-hub/aws-secrets-manager)**: Amazon Data Firehose can retrieve supported destination credentials from Secrets Manager. This lets delivery configurations reference a secret rather than embedding authentication credentials directly in the stream configuration.
- **[AWS Step Functions](https://blowstack.com/tools/cloud-services-hub/aws-step-functions)**: Amazon Data Firehose APIs can be invoked through Step Functions AWS SDK service integrations. A workflow can submit records to a delivery stream or manage streams as part of an orchestrated process.
- **[AWS Verified Access](https://blowstack.com/tools/cloud-services-hub/aws-verified-access)**: Amazon Data Firehose can receive Verified Access access logs through its logging configuration. Firehose delivers the records to a destination for analysis of application-access requests and authorization decisions.
- **[AWS WAF](https://blowstack.com/tools/cloud-services-hub/aws-waf)**: Amazon Data Firehose can receive AWS WAF web ACL traffic logs as a configured logging destination. Firehose buffers and delivers request and rule-evaluation records for security analysis.

## Amazon Data Firehose exam questions

Real practice exam questions on Amazon Data Firehose, with the correct answer and its explanation.

### Question 1 (CLF-C02)

**Question:**

A company’s applications continuously generate operational events. The company needs to deliver these events to Amazon S3 for later analysis, with a delivery delay of a few minutes being acceptable. It wants to buffer and compress the events automatically without writing a consumer application or creating an ETL job. Which solution best meets these requirements?

**Correct answer:** Send the events to Amazon Data Firehose with an Amazon S3 destination and enable compression.

**Explanation:** Amazon Data Firehose provides managed delivery of streaming data to S3. It can buffer and compress records before delivery without requiring a custom consumer application or an ETL job.

### Question 2 (MLA-C02 (beta))

**Question:**

A retailer collects JSON customer interaction events to build ML training datasets. Each event contains an email address that must be removed before the event is written to a curated Amazon S3 bucket. Compliance also requires every original, unmodified event to be retained in a separate, access-restricted S3 archive bucket. Delivery delays of several minutes are acceptable. Which solution meets these requirements with the least custom processing infrastructure?

**Correct answer:** Send events to Amazon Data Firehose. Use a Lambda transformation to remove email addresses before delivery to the curated bucket, and enable source-record backup of all incoming records to the separate archive bucket.

**Explanation:** Firehose can invoke Lambda to transform records before delivering them to S3. Source-record backup preserves the original, untransformed records in a separate S3 bucket, satisfying both the curated-data and archival requirements without a continuously running processing application.

### Question 3 (SAP-C03)

**Question:**

A company has 60 workload accounts in an AWS Organizations organizational unit (OU). Applications write logs to Amazon CloudWatch Logs in two AWS Regions. A separate log-archive account owns a central Amazon S3 bucket. The company must archive every newly ingested log event from existing and future workload log groups, with events normally reaching S3 within a few minutes. Logs must remain searchable in their source accounts for 30 days and archived in S3 for 7 years. Archived logs older than 90 days are rarely accessed, and a retrieval time of up to 48 hours is acceptable. The solution must automatically cover new workload accounts and must not require custom log-processing code.

Which solution meets these requirements with the LEAST ongoing operational effort?

**Correct answer:** In the log-archive account, create a CloudWatch Logs logical destination in each source Region, backed by a regional Amazon Data Firehose stream that delivers to the central S3 bucket. Configure destination access policies to authorize the workload accounts and service roles that allow delivery. Use an organization-integrated CloudFormation StackSet with automatic OU deployment to configure regional account-level subscription policies covering all workload log groups, the necessary sender permissions, and 30-day source retention. Configure S3 Lifecycle to transition objects to S3 Glacier Deep Archive after 90 days and expire them after 7 years.

**Explanation:** Regional logical destinations provide the supported cross-account subscription path, and Firehose provides managed, buffered delivery to S3 without custom processing code. Account-level subscription policies cover existing and future log groups. StackSets automatic deployment extends the configuration to new accounts in the OU. Source retention is independent of the S3 archive, and Glacier Deep Archive supports the stated retrieval window while reducing long-term storage costs.

### Question 4 (DEA-C01)

**Question:**

A company sends application events directly to an Amazon Data Firehose stream that delivers to Amazon S3. The producers generate a sustained 4,000 events per second, and each event is 1 KiB. Each event is currently submitted in a separate PutRecord request. The stream's currently applied ingestion quotas are 2,000 requests per second, 500,000 records per second, and 5 MiB per second. Monitoring shows request-rate throttling and a growing producer backlog.

The producers retain events durably until Firehose accepts them. The company must support the sustained event rate within the currently applied quotas, without adding streams or relying on capacity increases. Each event must remain a separate Firehose record. Up to 1 second of producer-side buffering is acceptable.

Which change should a data engineer make?

**Correct answer:** Use PutRecordBatch to submit up to 100 events per request, with each event supplied as a separate record. Flush pending batches within 1 second and retry failed records with exponential backoff and jitter.

**Explanation:** Batching 100 separate records reduces the required request rate to approximately 40 requests per second. The workload remains at 4,000 records per second and approximately 3.91 MiB per second, below the other applied quotas. These batches also fit within PutRecordBatch size limits. Inspecting batch results and retrying failed records handles transient failures without resubmitting records that were accepted.

### Question 5 (SCS-C03)

**Question:**

A company needs to send custom JSON security events from its applications to an external Splunk deployment through its HTTP Event Collector (HEC) endpoint. During normal operation, events must reach Splunk within a few minutes. Every event, including events successfully indexed by Splunk, must also be archived in Amazon S3 for one year. If Splunk remains unavailable beyond the delivery retry period, undelivered events must be preserved for later replay by the security team. The company wants managed buffering and delivery without maintaining custom forwarding code. Which solution meets these requirements?

**Correct answer:** Send the events to an Amazon Data Firehose delivery stream with Splunk as the destination. Enable HEC indexer acknowledgment, configure delivery retries, and enable S3 backup for all events. Configure the backup objects for one-year retention.

**Explanation:** Firehose provides managed buffering and delivery to Splunk, including HEC acknowledgment handling and delivery retries. Selecting backup for all events preserves both successful and unsuccessful deliveries in S3. Events that cannot be delivered within the retry period remain available in the backup for a separate replay process. Retaining these objects for one year satisfies the archive requirement.

### Question 6 (DVA-C02)

**Question:**

A developer is implementing an AWS Lambda transformation function for an Amazon Data Firehose stream that delivers device telemetry to Amazon S3. Each input record contains either a heartbeat or a measurement. The function must normalize measurement records before delivery and intentionally discard heartbeat records. Discarded heartbeats must not be treated as processing failures or written to the processing-failed S3 prefix. Which implementation meets these requirements?

**Correct answer:** Return an entry for every input record with its original recordId. For measurements, set result to Ok and include base64-encoded normalized data. For heartbeats, set result to Dropped and include the original base64-encoded data.

**Explanation:** The Firehose Lambda transformation contract uses recordId to associate each response with an input record. Ok identifies successfully transformed data, which must be base64-encoded. Dropped identifies intentional filtering and is treated as successful processing, so heartbeats are not delivered or routed to the processing-failed prefix.

### Question 7 (DOP-C02)

**Question:**

A company uses an Amazon Data Firehose delivery stream to deliver operational events to Amazon S3 for incident analysis. An AWS Lambda transformation function validates and normalizes each JSON event. Firehose data format conversion is disabled.

Each invocation can contain valid incident events, valid heartbeat events that must be discarded, and malformed events. The company requires valid incident events to reach the primary S3 prefix and malformed events to be retained in a separate S3 error prefix for investigation. Heartbeat events must not be delivered to either prefix. A malformed event must not cause successfully transformed events in the same invocation to be retried. The team wants to use Firehose's managed processing and error-delivery capabilities rather than implement separate S3 writes.

Which implementation meets these requirements?

**Correct answer:** Return Ok with normalized data for valid incident events, Dropped for heartbeat events, and ProcessingFailed for malformed events. Preserve the input recordId in every response entry, include the required base64-encoded data, and configure the Firehose S3 error output prefix.

**Explanation:** Firehose supports per-record transformation results. Ok records continue to the destination, Dropped records are intentionally excluded, and ProcessingFailed records are treated as unsuccessfully processed and delivered to S3 error storage. Returning a valid response for every input record isolates malformed records without failing the invocation or retrying successfully transformed records.

### Question 8 (SAA-C03)

**Question:**

A company sends telemetry directly to an Amazon Data Firehose delivery stream that delivers to Amazon S3. A scheduled equipment rollout will increase peak ingestion to 12 MiB/s, consisting of 24,000 Firehose records per second submitted through 600 PutRecordBatch requests per second. These measurements include all data counted against ingestion quotas.

The delivery stream currently has approved ingestion quotas of 5 MiB/s, 500,000 records per second, and 2,000 requests per second. Load testing confirms that transformation and destination capacity are sufficient. The company requires ingestion capacity at least 25% above the expected peak in every quota dimension. The required capacity must be available before the rollout because producers cannot tolerate ingestion throttling while capacity adjusts.

Which action should a solutions architect take?

**Correct answer:** Request an ingestion data-volume quota increase to 15 MiB/s, and confirm that the increase is applied before the rollout.

**Explanation:** The required data-volume capacity is 12 MiB/s × 1.25 = 15 MiB/s. The other requirements are 30,000 records per second and 750 requests per second, both below the existing quotas. Applying the quota increase before the rollout avoids depending on reactive capacity adjustments.

### Question 9 (ANS-C01)

**Question:**

A company stores VPC Flow Logs in Amazon CloudWatch Logs log groups in several application accounts. The security account has an Amazon Data Firehose delivery stream that sends records to a central audit bucket. All resources are in the same AWS Region. The security team requires continuous delivery of newly ingested flow log records through this existing delivery stream, without scheduled exports, custom forwarding code, or resources that application administrators must manage in the security account. Application accounts must retain their existing log groups. Which solution meets these requirements?

**Correct answer:** Create a CloudWatch Logs logical destination in the security account that targets the Firehose delivery stream and uses a role CloudWatch Logs can assume to write to the stream. Attach a destination access policy permitting the application accounts to create subscriptions. Grant the application administrators the required subscription permissions, and configure subscription filters on their log groups using the logical destination ARN.

**Explanation:** A CloudWatch Logs logical destination supports cross-account subscription delivery to a Firehose delivery stream. The receiving account supplies the destination, delivery role, and destination access policy. Authorized source accounts configure subscription filters referencing that destination. This continuously forwards new matching events while preserving the source log groups and keeping ownership of receiving resources in the security account.

## Practice Amazon Data Firehose in a full exam

Questions on Amazon Data Firehose in each practice exam, with every answer option and an explanation for each:

- [SAA-C03](https://blowstack.com/practice-exams/aws-certified-solutions-architect-associate): 13 questions
- [DOP-C02](https://blowstack.com/practice-exams/aws-certified-devops-engineer-professional): 8 questions
- [DEA-C01](https://blowstack.com/practice-exams/aws-certified-data-engineer-associate): 7 questions
- [MLA-C02 (beta)](https://blowstack.com/practice-exams/aws-certified-machine-learning-engineer-associate): 4 questions
- [ANS-C01](https://blowstack.com/practice-exams/aws-certified-advanced-networking-specialty): 4 questions
- [CLF-C02](https://blowstack.com/practice-exams/aws-certified-cloud-practitioner): 2 questions
- [SAP-C03](https://blowstack.com/practice-exams/aws-certified-solutions-architect-professional): 2 questions
- [SCS-C03](https://blowstack.com/practice-exams/aws-certified-security-specialty): 2 questions
- [DVA-C02](https://blowstack.com/practice-exams/aws-certified-developer-associate): 2 questions

## What is Amazon Data Firehose

Amazon Data Firehose is a fully managed service that facilitates the effortless streaming of data in real-time to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon OpenSearch Service, and Splunk, allowing for near real-time analytics with existing business intelligence tools and dashboards.

It's part of the broader Amazon Kinesis suite, which provides solutions to collect, process, and analyze real-time, streaming data, enabling developers and businesses to gain timely insights and react promptly to new information. The process of utilizing Amazon Data Firehose begins with the creation of a Firehose delivery stream. Once created, you can send your streaming data, such as application logs, website clickstreams, IoT telemetry data, and more to the Firehose service, which then automatically delivers the data to the specified destination.

Firehose buffers incoming data before delivery using a buffer-size threshold and a buffer interval measured in seconds.

PutRecordBatch accepts up to 500 records per request, with each record limited to 1,000 KB before Base64 encoding and the entire request limited to 4 MB. These limits cannot be changed.

Bursts of incoming bytes or records lasting only a few seconds might not be fully visible in one-minute CloudWatch metrics.

One of the key features of Firehose is its ability to transform and prepare data before delivering it. This includes tasks like converting data formats to match the target storage solution, compressing the data to reduce storage costs, and even encrypting the data for enhanced security before it lands in the destination service. Data transformation is facilitated through integration with AWS Lambda, allowing customization of the data as it flows through Firehose, making it extremely adaptable to various data processing needs. Through this integration, data analytics and transformation can be deployed without managing any additional infrastructure, further simplifying the architecture required for real-time analytics. Amazon Data Firehose manages all the resources required to operate and automatically scales to match the throughput of your data, making it incredibly scalable.

When dynamic partitioning is enabled, a Firehose stream has a default quota of 500 active partitions. Active partitions are those currently held in the delivery buffer; a partition stops being active once its data is delivered.

It can handle any amount of data, from small to colossal streams, without upfront cost or the need to provision resources manually. This level of scalability and management frees up developers and data scientists to focus on analyzing data rather than managing backend infrastructure. Billing for Amazon Kinesis Data Firehose is straightforward and is based on the amount of data ingested into the service. This pay-as-you-go model ensures that costs are directly aligned with usage, offering an efficient way to manage expenses related to streaming data analytics.

In summary, Amazon Data Firehose provides a robust, scalable, and manageable solution for streaming large volumes of data in real-time to various destinations. Its ability to transform data on the fly, coupled with automatic scaling and simple yet powerful integration points, makes it an indispensable tool for businesses looking to leverage their data for timely insights and decision-making.

## Learn Amazon Data Firehose in lessons

15 short lessons on Amazon Data Firehose in 7 certification courses.

- CLF-C02: Amazon Data Firehose: managed streaming data delivery (Premium), https://app.blowstack.com/lessons/amazon-data-firehose-managed-streaming-data-delivery-clf-c02
- MLA-C02 (beta): Amazon Kinesis Data Firehose: managed streaming delivery (Premium), https://app.blowstack.com/lessons/amazon-kinesis-data-firehose-managed-streaming-delivery-mla-c02
- SAP-C03: Amazon Data Firehose: centralized log delivery to Amazon S3 (Premium), https://app.blowstack.com/lessons/amazon-data-firehose-centralized-log-delivery-to-amazon-s3-sap-c03
- DEA-C01: Amazon Kinesis Data Firehose: record format conversion (Premium), https://app.blowstack.com/lessons/amazon-kinesis-data-firehose-record-format-conversion-dea-c01
- SCS-C03: Amazon Kinesis Data Firehose: reliable log delivery (Premium), https://app.blowstack.com/lessons/amazon-kinesis-data-firehose-reliable-log-delivery-scs-c03
- DOP-C02: Amazon Kinesis Data Firehose: delivery for CloudWatch metric streams (Premium), https://app.blowstack.com/lessons/amazon-kinesis-data-firehose-delivery-for-cloudwatch-metric-streams-dop-c02

## Concepts related to Amazon Data Firehose

- [Partitioning](https://blowstack.com/tools/cloud-concepts-hub/partitioning): Partitioning divides data or a stream of records into smaller, addressable groups so storage, ingestion, and processing can scale efficiently.
- [Fan-In](https://blowstack.com/tools/cloud-concepts-hub/fan-in): Fan-in is a many-to-one data-flow pattern in which multiple producers or streams feed a shared destination or processing stage.
- [JSON](https://blowstack.com/tools/cloud-concepts-hub/json): JSON is a text-based data interchange format that represents structured values using objects, arrays, strings, numbers, booleans, and null.
