Cloud concept · Updated

Batching - Batching groups multiple records, requests, or work items into a unit for more efficient transfer, storage, or processing.

In short

Batching spreads fixed overhead across multiple items instead of handling every item independently. It can reduce request costs and improve ingestion throughput, but introduces buffering delays and requires careful handling of failures, retries, and resource limits.

Cloud Concepts Hub →

6 key aspects of Batching

Effective batching balances lower per-item overhead against latency, resource consumption, storage access patterns, and recovery requirements.

01
Amortized overhead

Handle multiple items per transfer or execution to spread connection, request, and scheduling overhead across more useful work. Confirm which billing dimensions actually decrease; fewer requests do not necessarily mean fewer metered records or bytes.

02
Size- and time-based flushing

Flush when a record-count or byte-size threshold is reached, or when a maximum waiting interval expires. This balances efficiency during busy periods with acceptable latency during quiet periods.

03
Storage granularity

Combine small records into appropriately sized objects when the access pattern permits it. Preserve useful partition boundaries and choose a format that supports downstream queries rather than creating oversized, difficult-to-read archives.

04
Throughput and resource tuning

Tune batch size alongside concurrency, memory, payload limits, and destination capacity. Excessively large batches can cause timeouts, memory pressure, or bursts that overwhelm downstream systems.

05
Failure isolation and safe retries

Inspect item-level outcomes and retry only failed items where supported. Make processing idempotent because retries and redelivery can repeat work, and do not treat a batch as an implicit transaction.

06
Durable buffering

Protect records while they wait to be flushed by using persistent buffers or durable queues when required. Monitor record age and backlog so cost savings do not hide unacceptable delivery delays.

Cloud services for Batching

A producer can batch records into Kinesis Data Streams or send messages using SQS batch APIs, while Lambda processes records in groups through supported event source mappings. For buffered delivery to S3, Data Firehose can accumulate records before writing objects. Glue can subsequently transform the stored data and compact small files for analytics. These are different batching boundaries: reducing producer API calls does not necessarily reduce S3 object counts, and reducing function invocations does not guarantee lower total processing cost. Coordinate buffer intervals, batch sizes, concurrency, and retry behavior across the pipeline to meet both cost and freshness requirements.

Amazon S3Amazon S3

Producers can combine small records into fewer objects before uploading to reduce object write requests and small-file overhead. Uploading separate objects together still requires separate object writes; multipart upload and S3 Batch Operations serve different purposes.

Amazon Kinesis Data StreamsAmazon Kinesis Data Streams

The PutRecords API submits multiple records in one request, reducing producer request overhead without removing stream capacity constraints. Inspect individual record results and retry failures carefully, particularly when ordering matters.

AWS LambdaAWS Lambda

Event source mappings for supported queues and streams deliver batches of records to function invocations. Batch size, supported batching windows, concurrency, and source-specific failure handling affect throughput, latency, and retry scope.

AWS GlueAWS Glue

Glue ETL jobs can process accumulated datasets and compact small files into larger analytical files. Its streaming ETL jobs use micro-batches, with processing intervals chosen to balance data freshness and processing efficiency.

Batching exam questions

Real practice exam questions on Batching, with the correct answer and its explanation. Answer them with every option in the exam simulator.

DVA-C02

A developer is optimizing a background import worker that writes millions of independent, 1-KB items to Amazon DynamoDB. The worker currently calls PutItem once for each item. All items have distinct primary keys, and the import requires neither conditional writes nor all-or-nothing transactions. Profiling shows that per-request network overhead limits throughput. The table occasionally throttles some writes during traffic bursts.

The developer must reduce the number of write API requests without increasing the write capacity required per item. The worker must also complete partially successful requests without rewriting items that DynamoDB has already processed.

Which implementation meets these requirements?

Show answerCORRECT ANSWER

Send up to 25 put operations in each BatchWriteItem request. Retry only the operations returned in UnprocessedItems, using exponential backoff.

BatchWriteItem combines up to 25 independent put or delete operations into one API request, reducing per-request overhead. Each write consumes the same write capacity as an individual nontransactional write. Retrying only UnprocessedItems with exponential backoff completes outstanding work without repeating successful writes.

SAP-C03

A financial services company uses a custom GPU-based model in Amazon SageMaker AI to classify 12 million transaction records each night. All records are available in Amazon S3 before processing begins, and predictions must be written to S3 within 4 hours. No application needs interactive predictions. The company currently keeps a real-time endpoint running continuously and invokes it separately for each record. Profiling shows that request-handling overhead dominates processing time. The inference container supports newline-delimited JSON and can process multiple records in a single invocation substantially more efficiently than individual records. The company must retain the existing model and prediction accuracy. Which solution will MOST cost-effectively meet these requirements?

Show answerCORRECT ANSWER

Create a SageMaker Batch Transform job each night using appropriately sized GPU instances. Use balanced newline-delimited input files, set SplitType to Line and BatchStrategy to MultiRecord, and tune payload size, concurrency, and instance count to meet the deadline.

Batch Transform fits an offline workload whose inputs and outputs are in S3 and provisions inference resources for the job rather than maintaining an idle endpoint. MultiRecord allows multiple records in each invocation, enabling the container's efficient batched processing. Balanced files and tested payload, concurrency, and instance settings help meet the processing deadline at lower cost.

SAA-C03

A company uploads finalized 64-GiB research archives to Amazon S3 Standard. Each archive must remain a separate object. The uploader currently uses multipart upload with 8-MiB parts, and S3 upload request charges have become significant. Performance tests show that using either 8-MiB parts or 128-MiB parts meets the upload deadline and stays within the uploader's memory limits. The company must retain the ability to retry failed portions without restarting an entire archive upload.

Which change will reduce S3 upload request charges the MOST while meeting these requirements?

Show answerCORRECT ANSWER

Increase the multipart upload part size to 128 MiB while continuing to upload each archive as a separate object.

Each uploaded part incurs a request charge. Increasing the part size from 8 MiB to 128 MiB reduces the number of part upload requests for each archive from 8,192 to 512. Multipart upload still supports retrying individual failed parts, and testing confirms that the larger parts meet the performance and memory requirements.

Practice Batching in a full exam

Questions on Batching in each practice exam, with every answer option and an explanation for each:

Start a free practice exam

What is Batching

3 min read

Batching is the practice of collecting multiple items before transferring or processing them together. Each network request, storage write, or processing invocation has overhead, such as connection handling, authentication, scheduling, and metadata operations. Sharing that overhead across many items can improve throughput and reduce cost per item. Batching is useful for logs, telemetry, application events, database updates, and analytical data ingestion when immediate processing of every individual item is unnecessary.

A common implementation uses both a size threshold and a time threshold. A producer might flush its buffer when it reaches a configured number of records or bytes, or when the oldest record has waited a specified interval, whichever happens first. The size threshold supports efficient processing during heavy traffic, while the time threshold prevents records from waiting indefinitely during quiet periods. Choose these values according to the application's freshness requirement, destination limits, available memory, and downstream processing capacity.

For object storage, combining many small records into fewer larger objects can reduce upload request counts and downstream file-opening overhead. For example, an application could collect log events into compressed files before uploading them to Amazon S3 rather than creating one object per event. Compression can additionally reduce stored and transferred bytes, although its effectiveness depends on the data and format. Combining records also changes retrieval granularity: applications may need to read or parse part of a larger object to retrieve one event.

Distinguish combining records from merely scheduling many uploads together. Uploading 1,000 separate files still requires object write requests for those files; running the uploads together does not turn them into one billable PUT request. Amazon S3 does not provide a general multi-object PUT operation. Multipart upload is also different: it transfers one large object in parts, often concurrently, and is not a way to combine separate objects into one request. S3 Batch Operations manages supported actions across many existing objects rather than providing a discounted bulk-upload API.

In ingestion pipelines, batched producer calls can carry multiple records, and consumers can process several records per invocation. These mechanisms can reduce API calls and invocation overhead, but they do not automatically reduce every charge. Some services meter individual records, rounded record sizes, bytes, or processing duration. Batching also does not remove destination throughput limits. Measure sustained ingestion rate, throttling, processing duration, buffer occupancy, and end-to-end record age rather than assuming the largest supported batch is optimal.

Larger batches increase memory requirements, can create bursty downstream load, and may enlarge the amount of work affected by a failure. A successful HTTP response from a batch API does not necessarily mean every item succeeded; inspect item-level results where the API provides them. Retry failed items with backoff when supported, and use idempotent processing or deduplication to prevent duplicate effects. Do not assume a batch is an atomic transaction or that it preserves ordering unless the specific interface guarantees those properties.

Buffer durability matters because records waiting only in application memory can be lost if the producer crashes. Use persistent buffering or a durable intermediary when losing accepted records is unacceptable. For architecture decisions, prefer batching when the workload tolerates some delay and fixed per-request overhead is significant. Prefer smaller batches or immediate handling for latency-sensitive events, and tune batch size, buffering interval, and concurrency together through representative load tests.

Learn Batching in lessons

2 short lessons on Batching in 2 certification courses.

Open the lessons