Cloud concept · Updated

Throughput - Throughput is the amount of data or work a system successfully processes per unit of time.

In short

Throughput measures the rate at which storage, networks, applications, and data pipelines deliver results, commonly expressed as bytes, operations, or records per second. It matters when selecting storage configurations, sizing ingestion and transformation workloads, and determining whether a system can meet processing deadlines without accumulating a backlog.

Cloud Concepts Hub →

6 key aspects of Throughput

Practical throughput design combines measurable completion rates, workload-aware storage selection, balanced pipeline capacity, and sustained performance testing.

01
Explicit units and measurement boundaries

Define whether the target is bytes, records, operations, or transactions per second, and measure successful completion at a specified point. Distinguish decimal MB/s from binary MiB/s when calculating capacity.

02
Workload-dependent storage performance

Evaluate I/O size, sequential versus random access, read/write mix, and concurrency together. High IOPS alone does not guarantee sufficient byte throughput for large scans or bulk transfers.

03
End-to-end bottleneck analysis

Check application workers, storage limits, instance capacity, networks, partitions, and destinations. Increasing capacity at a stage that is not the bottleneck usually does not improve overall throughput.

04
Deadline and backlog sizing

Divide data volume by the available processing window to establish a minimum rate, then allow headroom for overhead and variability. For streams, provision enough processing capacity to handle incoming traffic and recover from accumulated backlog.

05
Parallelism, partitioning, and batching

Use concurrent requests, balanced partitions, and appropriately sized batches to use available capacity efficiently. Watch for hot partitions, ordering requirements, downstream throttling, and increased per-record latency.

06
Sustained performance and cost

Separate temporary burst capacity from performance that can be maintained for the full workload duration. Benchmark representative workloads and compare the cost of meeting the required completion rate rather than selecting the highest advertised rate.

Cloud services for Throughput

These services apply throughput at different points in a data architecture. Amazon EBS supports block-storage workloads, Amazon EFS supports shared-file workloads, and Amazon S3 provides object storage for ingestion and analytics. Kinesis Data Streams can buffer streaming input, while AWS Glue can transform supported streaming or stored datasets into downstream outputs. Size each stage using consistent units and realistic data characteristics. Monitor storage transfer rates, ingestion rates, processing completion rates, and consumer lag together so that a faster producer or larger transformation job does not simply shift the bottleneck to another stage.

Amazon S3Amazon S3

Amazon S3 supports high-throughput object ingestion and retrieval through concurrent requests and multipart uploads for large objects. Client network capacity, request patterns, object sizes, and processing capacity determine the useful rate achieved by an application.

Amazon Kinesis Data StreamsAmazon Kinesis Data Streams

Kinesis Data Streams ingests and exposes streaming records for consumers, with capacity constraints that include both record rates and byte rates. Capacity mode, partition-key distribution, and consumer read capacity affect whether the stream can sustain arrivals without increasing lag.

AWS GlueAWS Glue

AWS Glue runs data integration and transformation workloads whose throughput depends on worker resources, parallelism, data layout, and transformation complexity. Size and tune jobs to meet processing windows while checking for skew, excessive small files, and source or destination bottlenecks.

Throughput exam questions

Real practice exam questions on Throughput, with the correct answer and its explanation. Answer them with every option in the exam simulator.

MLA-C02 (beta)

A company continuously uploads training-data files to an Amazon S3 general-purpose bucket. Every object must remain encrypted with the same customer managed AWS KMS key. All uploads use one ingestion IAM role.

As ingestion volume grows, the upload backlog increases. Investigation shows that AWS KMS is throttling GenerateDataKey requests made by S3 for the uploads. Network capacity and upload-worker resources are sufficient, and S3 request rates remain below its scaling limits. The application already retries throttled requests with exponential backoff, but the sustained arrival rate exceeds the throughput supported by the current encryption request pattern.

The company cannot increase its KMS request quota or change the required KMS key. Which change should an ML engineer make to improve sustained ingestion throughput?

Show answerCORRECT ANSWER

Enable S3 Bucket Keys for new uploads while continuing to use the required customer managed KMS key.

S3 Bucket Keys reduce the number of requests S3 makes to KMS by using a time-limited bucket-level key to create unique object data keys. This addresses the identified KMS throughput bottleneck while preserving SSE-KMS encryption with the required customer managed key.

SAP-C03

A company operates a fleet of Amazon EC2 workers that process large scientific datasets. Each worker requires 6 TiB of scratch storage on a single Amazon EBS data volume. The workload performs large, sequential reads and writes continuously for 18 hours each day. Testing establishes a requirement for sustained throughput of 300 MiB/s, including performance headroom. The solution must not depend on accumulated burst credits. HDD latency is acceptable, and the data can be regenerated if necessary.

Each EC2 instance supports sustained EBS throughput of 1,000 MiB/s. The application generates enough concurrent I/O to achieve the required throughput. On SSD volumes, the workload requires no more than 3,000 EBS IOPS.

In the company's AWS Region, st1 storage costs $0.045 per GiB-month. gp3 storage costs $0.08 per GiB-month, including 3,000 IOPS and 125 MiB/s of throughput. Additional gp3 throughput costs $0.04 per MiB/s-month. The company uses On-Demand pricing without discounts.

Which proposed configuration meets the requirements at the LOWEST monthly EBS cost per worker?

Show answerCORRECT ANSWER

Use one 8 TiB st1 volume.

st1 provides baseline throughput of 40 MiB/s per TiB, so an 8 TiB volume provides 320 MiB/s without relying on burst credits. It meets the capacity and sustained-throughput requirements for $368.64 per month (8 × 1,024 × $0.045), less than the qualifying gp3 configuration. Paying for extra HDD capacity is cheaper here than using SSD storage with additional provisioned throughput.

DVA-C02

A document-processing application runs a fleet of workers. Requests are accepted quickly, but the backlog grows during peak traffic. Each worker commits results in batches of varying sizes. After each successful commit, the worker publishes one datum to an Amazon CloudWatch custom metric named DocumentsCommitted. The datum's value is the number of documents in that batch. Each document is counted exactly once. All workers publish to the same metric with identical dimensions.

The application also publishes a DocumentsAccepted metric that counts documents entering the pipeline. Metrics are delivered promptly, and every one-minute period contains multiple completed batches.

A developer needs an alarm that detects when the entire fleet's successful output falls below 2,000 documents per second for five consecutive one-minute periods while a backlog exists. Which calculation should the alarm compare with 2,000 for each period?

Show answerCORRECT ANSWER

Use the Sum statistic for DocumentsCommitted with a 60-second period, divided by 60.

Sum adds the document counts from all successful batches across the fleet during the minute. Dividing by 60 converts that total into successfully committed documents per second, which is the required output throughput.

Practice Throughput in a full exam

Questions on Throughput in each practice exam, with every answer option and an explanation for each:

Start a free practice exam

What is Throughput

3 min read

Throughput describes completed work over a measured interval. For storage and data movement, it is commonly measured in MB/s, MiB/s, or GB/s; for applications and pipelines, it can be measured in requests, transactions, or records per second. Always identify what is being counted and where it is measured. A producer's attempted submission rate is not necessarily the rate successfully accepted by an ingestion service or committed to destination storage.

Throughput differs from latency and bandwidth. Latency measures the time required for an individual operation, while throughput measures how much work completes over time. Bandwidth describes a communication channel's capacity, not necessarily the application throughput achieved over that channel. Parallel operations can increase aggregate throughput without reducing individual operation latency. Excessive concurrency, however, can create contention and queueing that increase latency without improving completion rates.

Storage sizing requires understanding both bytes per second and input/output operations per second, or IOPS. Approximately, data throughput equals completed IOPS multiplied by average I/O size, provided those measurements describe the same workload and measurement boundary. For example, 10,000 operations per second at 16 KiB per operation represent about 156 MiB/s. Small random database reads often encounter an IOPS or latency limit, while large sequential scans often encounter a data-throughput limit. A configuration suitable for one access pattern may perform poorly for the other.

Effective storage performance depends on the entire path. A volume's configured throughput cannot overcome a lower instance-level storage limit, insufficient network capacity, or an application issuing too little concurrent I/O. Shared file systems also depend on client behavior, metadata activity, and the service's performance configuration. Object-storage workloads often benefit from concurrent requests and multipart uploads for large objects, but excessive small objects can increase request overhead and transformation costs.

Translate business requirements into minimum sustained rates before selecting resources. Reading 1 TiB within one hour requires approximately 291 MiB/s of useful input throughput, before allowing for startup time, retries, protocol overhead, or processing delays. If transformation expands the data, the output path may need greater throughput than the input path. If it compresses the data, reduced network and storage traffic comes at the cost of additional computation.

For streaming ingestion, consider both record count and record size. An arrival rate of 20,000 records per second at 2 KiB each represents approximately 39 MiB/s of payload. Partition distribution, per-partition limits, producer batching, and consumer capacity can all constrain the achieved rate. Uneven partition keys can create a bottleneck even when aggregate capacity appears sufficient. Size the transformation stage using representative records because parsing complexity, joins, enrichment calls, and serialization affect processing speed.

A pipeline's sustained output is constrained by its bottleneck, accounting for changes in data volume between stages. If producers deliver work faster than consumers complete it, buffering absorbs the difference temporarily but does not eliminate the capacity deficit. Consumers need capacity above the ongoing arrival rate to clear an existing backlog. Monitor queue depth or consumer lag alongside successful processing rates to distinguish healthy buffering from an increasingly delayed pipeline.

Higher throughput usually requires additional cost or operational trade-offs, such as more workers, higher storage performance settings, larger batches, or increased parallelism. Larger batches improve efficiency but can delay individual records. Burst performance is not a substitute for sustained capacity during long-running jobs. Validate designs with realistic data sizes, access patterns, and peak loads, and measure useful completed work rather than counting retries as additional progress.

Learn Throughput in lessons

6 short lessons on Throughput in 3 certification courses.

Open the lessons