Cloud concept · Updated

Latency - Latency is the elapsed time between a defined starting event and the corresponding response, completion, or availability of a result.

In short

Latency measures delay rather than how much work a system completes per second. In cloud data pipelines, it determines how quickly ingested data becomes useful, so architecture and capacity must match business deadlines for processing, delivery, and freshness.

Cloud Concepts Hub →

6 key aspects of Latency

Effective latency design combines clear timing boundaries, workload-aware capacity planning, controlled buffering, and end-to-end percentile measurements.

01
Explicit measurement boundaries

Define the start and finish of every latency target, such as event creation to query availability. Separate ingestion acknowledgment, processing completion, and end-to-end freshness so that a fast intermediate step does not hide a slow pipeline.

02
Latency versus throughput

Measure elapsed time separately from records or bytes processed per second. High throughput does not guarantee fast individual results, especially when batching or queueing is involved.

03
Capacity matched to workload size and speed

Size each stage for peak record rates, byte rates, payload sizes, and transformation costs. Maintain enough headroom to handle bursts and clear backlog within the business deadline.

04
Buffering and batching trade-offs

Tune batch sizes and buffering intervals against the required result deadline. Larger batches can reduce per-record overhead, while smaller batches generally reduce waiting time at the cost of more frequent operations.

05
Percentiles and tail behavior

Use percentile targets such as p95 or p99 alongside error rates and throughput. Investigate slow outliers caused by retries, resource contention, initialization, throttling, or uneven partition traffic.

06
End-to-end bottleneck analysis

Break the latency budget into transport, queueing, processing, and destination stages. Optimize the limiting stage rather than assuming that more compute or faster networking will improve the entire workflow.

Cloud services for Latency

Kinesis Data Streams can ingest events for processing by Lambda or Managed Service for Apache Flink when results must be produced continuously. Data Firehose provides a managed delivery path where buffering-related delay is acceptable, while AWS Glue supports scheduled batch transformations and streaming integration. These are alternative or complementary building blocks, not a requirement to use every service in one pipeline. CloudWatch helps correlate processing duration, throttling, and backlog across the selected components. The design must also measure when data becomes usable at its destination, because fast ingestion alone does not demonstrate that the business latency target has been met.

Amazon Kinesis Data StreamsAmazon Kinesis Data Streams

Supports continuous event ingestion and consumption for latency-sensitive pipelines. Stream capacity, partition-key distribution, consumer read behavior, and processing capacity affect how quickly records reach downstream results.

Amazon Managed Service for Apache FlinkAmazon Managed Service for Apache Flink

Runs continuous stream transformations and stateful computations. Parallelism, backpressure, window definitions, and event-time handling affect processing delay and when results can be emitted.

AWS LambdaAWS Lambda

Provides event-driven processing without managing servers. For stream-triggered functions, batch configuration, execution duration, concurrency, and retries influence record processing latency and backlog.

AWS GlueAWS Glue

Provides batch and streaming data integration capabilities. Choose processing mode, worker capacity, and scheduling or micro-batch settings to meet freshness requirements while accounting for startup, transformation, and write time.

Amazon CloudWatchAmazon CloudWatch

Collects metrics and logs used to investigate pipeline delay and capacity bottlenecks. Combine service metrics with application-level event timestamps to measure end-to-end latency rather than relying only on component duration.

Latency exam questions

Real practice exam questions on Latency, with the correct answer and its explanation. Answer them with every option in the exam simulator.

DVA-C02

A background worker on Amazon EC2 occasionally experiences queue-wait spikes that last only a few seconds. The application already publishes one QueueWaitTime sample each second as a standard-resolution custom metric in Amazon CloudWatch. One-minute metric charts obscure when the spikes occur.

A developer must use CloudWatch metric charts to examine the maximum sampled queue-wait time in each 10-second interval during the next load test.

Which solution meets this requirement?

Show answerCORRECT ANSWER

Publish the custom metric with StorageResolution set to 1. Continue publishing once each second and configure the chart with a 10-second period and the Maximum statistic.

Setting StorageResolution to 1 creates a high-resolution custom metric. CloudWatch can display it using a 10-second period, and Maximum shows the greatest sampled queue-wait time within each interval.

ANS-C01

A financial services company sends UDP market data from its data center to EC2 receivers through AWS Direct Connect. The forward and return paths traverse different routers, and their delays can change independently.

The network team needs a CloudWatch dashboard that displays the absolute one-way network transit time of representative probes from the data center to AWS, including sustained delay after a routing change. Measurements must exclude application processing time and distinguish changes of approximately 2 milliseconds. Monitoring software can be deployed at both endpoints.

Which monitoring design meets these requirements?

Show answerCORRECT ANSWER

Synchronize and continuously verify the endpoint clocks so their relative error remains well below 2 milliseconds. Use sequence-numbered representative UDP probes with transmit and receive timestamps captured at the network interfaces. Publish the matched receive-minus-transmit times as a custom CloudWatch metric.

Matched timestamps at the transmitting and receiving network interfaces measure the required one-way transit interval without including return-path delay or application processing. Verified clock accuracy and sufficiently precise interface timestamping allow the measurements to distinguish changes of approximately 2 milliseconds and show sustained absolute delay.

SAA-C03

A company ingests sensor events into an Amazon Kinesis Data Streams stream with four provisioned shards. An AWS Lambda function transforms the events and writes them to a database. During sustained peaks, each shard receives approximately 250 records per second, with each record approximately 1 KB. Records are distributed evenly across many partition keys and can be transformed independently.

The Lambda event source mapping has a batch size of 100, a maximum batching window of 0 seconds, and a parallelization factor of 1. Each full batch takes approximately 1 second to process. Iterator age steadily increases during peaks, although ingestion acknowledgments remain fast. There is no read or write throttling, the database has spare capacity, and the function has enough available concurrency to process 16 batches simultaneously.

The company needs transformed records to reach the database within approximately 5 seconds of ingestion during normal peak operation. Which change should a solutions architect make?

Show answerCORRECT ANSWER

Increase the event source mapping's parallelization factor from 1 to 4.

At a parallelization factor of 1, each shard processes approximately 100 records per second but receives 250 records per second, so a backlog grows. A factor of 4 allows concurrent batches from each shard, providing approximately 400 records per second of processing capacity per shard. The many independent partition keys and sufficient concurrency allow this parallelism to reduce the backlog and meet the latency target.

Practice Latency in a full exam

Questions on Latency in each practice exam, with every answer option and an explanation for each:

Start a free practice exam

What is Latency

2 min read

Latency describes how long an operation takes, but the measurement is meaningful only when its boundaries are explicit. API latency might run from sending a request to receiving its response. In a data pipeline, end-to-end latency might run from an event occurring at its source to the transformed record becoming queryable. An ingestion acknowledgment can arrive quickly even when downstream processing takes minutes, so acknowledgment time must not be confused with time to a usable result.

Translate business requirements into measurable targets before selecting infrastructure. A fraud-detection workflow might require a decision within 200 milliseconds, while an operational dashboard might tolerate data that is 30 seconds old. Overnight reporting may only require completion before a morning deadline. These requirements lead to different choices among synchronous processing, continuous stream processing, micro-batches, and scheduled batch jobs.

Latency and throughput are related but distinct. Throughput measures records, requests, or bytes processed per unit of time; latency measures elapsed time for an individual operation or event. A system can process a large volume efficiently while making every record wait in a batch. Increasing network bandwidth can shorten large transfers, but it does not eliminate propagation delay, connection establishment, queueing, or transformation time.

Size ingestion and transformation capacity using both record counts and data volume. For example, 10,000 records per second at an average of 2 KB per record represent approximately 20 MB per second of payload before protocol overhead. Check peak rates, maximum record sizes, compression, partition distribution, and transformation complexity as well as averages. Small records can exhaust request or record-rate limits, while large records can exhaust byte-rate limits. A processing stage that cannot sustain arrivals accumulates backlog, increasing latency even when ingestion continues successfully.

End-to-end delay includes producer buffering, network transit, service queueing, processing, retries, and destination write or visibility time. Measure these stages separately to find the actual bottleneck. Place frequently communicating components near one another where resilience and compliance requirements permit, reuse connections, avoid unnecessary data movement, and parallelize independent work. Partitioning enables concurrency, but a hot partition or a strict global ordering requirement can limit the benefit of additional workers.

Batching illustrates an important trade-off. Larger batches amortize request overhead and can improve throughput and cost efficiency, but records wait for the batch to fill or its timer to expire. Smaller batches reduce that waiting period but increase request volume and may reduce processing efficiency. Similarly, asynchronous queues absorb bursts and decouple components, but they do not guarantee low completion latency. Consumers need sufficient spare capacity to drain accumulated work.

Evaluate latency distributions rather than averages alone. A p99 latency of 500 milliseconds means approximately 99 percent of measured operations finish within that time; slower outliers still exist. Test with realistic payloads, burst traffic, downstream throttling, and failures. Monitor event age, backlog, processing duration, and destination availability together. Autoscaling, retries, and additional concurrency can help, but scaling delays, duplicate processing, and downstream capacity constraints must remain part of the design.

Learn Latency in lessons

2 short lessons on Latency in 1 certification course.

Open the lessons