Cloud concept · Updated

Exponential Backoff - Exponential backoff is a retry strategy that increases the delay between successive attempts, usually by multiplying the previous delay by a fixed factor.

In short

Exponential backoff gives an overloaded or temporarily unavailable dependency time to recover instead of receiving immediate repeated requests. In cloud systems, it is typically combined with randomized delays, retry limits, and safe request semantics to handle throttling and transient failures without amplifying an outage.

Cloud Concepts Hub →

6 key aspects of Exponential Backoff

Effective exponential backoff combines increasing delays, jitter, selective and safe retries, bounded retry budgets, and corrective action for persistent capacity problems.

01
Exponentially increasing, capped delays

Increase the retry delay by a fixed multiplier after each failure and cap it at a practical maximum. Also limit total attempts and elapsed time so that a capped delay does not become an infinite retry loop.

02
Jitter to spread retry traffic

Randomize retry delays so clients do not repeatedly reconnect or resend at the same instant. Full jitter samples from zero to the current exponential delay ceiling.

03
Selective retries

Retry documented transient failures and throttling conditions, not every error. Inspect batch responses for item-level failures even when the overall HTTP request succeeds.

04
Idempotency and ambiguous outcomes

Treat timeouts as potentially successful operations whose responses were lost. Use idempotency tokens, deduplication, conditional updates, or transaction-aware retry logic to avoid duplicate effects.

05
Coordinated retry budgets

Account for retries in SDKs, application code, workflow tasks, and job orchestration. Keep the combined attempts and waiting time within the caller's deadline and avoid multiplying requests across layers.

06
Capacity-aware recovery

Use backoff alongside rate limits, concurrency controls, and appropriate capacity. Sustained throttling requires investigating quotas, hot partitions, connection limits, or workload demand rather than merely increasing retry counts.

Cloud services for Exponential Backoff

API Gateway can limit incoming request rates while Lambda or Step Functions coordinates downstream work. SDK retries handle recognized transient API failures, and workflow retries provide a separate recovery boundary when configured deliberately. DynamoDB and Kinesis Data Streams require additional application logic for partial batch failures so successful work is not unnecessarily repeated. For Amazon Bedrock, the same retry discipline must account for token quotas, generation latency, streaming output, and invocation costs. Across these services, coordinate retry limits, preserve idempotency, and monitor throttling and accumulated delay. Persistent overload should trigger concurrency reductions, capacity changes, or graceful degradation rather than an expanding chain of retries.

Amazon DynamoDBAmazon DynamoDB

Clients use exponential backoff for retryable throttling failures. BatchWriteItem and BatchGetItem can return unprocessed work in successful responses, so application code must inspect those results and retry only the unprocessed portion.

AWS Step FunctionsAWS Step Functions

Workflow retry policies can configure IntervalSeconds, BackoffRate, MaxAttempts, MaxDelaySeconds, and JitterStrategy for supported retry configurations. Use targeted error matching and Catch handling to recover when retries are exhausted.

Amazon Kinesis Data StreamsAmazon Kinesis Data Streams

Producers can back off when shard or stream throughput limits cause retryable write failures. PutRecords requires inspecting per-record results; retrying failed records can affect ordering and does not eliminate duplicate-delivery risks.

Amazon Bedrock

Applications invoking foundation models use bounded retries for documented transient failures and throttling. Coordinate retries with request and token quotas, latency budgets, streaming behavior, and the possibility that a timed-out invocation has already performed billable work.

AWS LambdaAWS Lambda

Function code can use SDK retry policies when calling downstream APIs, with timeouts and attempts constrained by the remaining invocation time. Invocation-level retries depend on the invocation source and must be coordinated with in-function retries and idempotency.

Amazon API GatewayAmazon API Gateway

API Gateway applies request throttling and can return throttling responses that clients handle with backoff and jitter. Gateway request-rate controls do not automatically enforce a foundation model's token budget or provide a universal backend retry policy.

Exponential Backoff exam questions

Real practice exam questions on Exponential Backoff, with the correct answer and its explanation. Answer them with every option in the exam simulator.

DEA-C01

A data ingestion application uses multiple Python workers and Boto3 to write records to Amazon DynamoDB with batch_write_item. During brief traffic bursts, calls return HTTP 200 responses with some records listed in UnprocessedItems. The workers currently treat these responses as complete, leaving records missing from the table.

The application must retry the unprocessed writes without resubmitting writes that DynamoDB already accepted. The retry strategy must spread retries across workers, reduce request frequency when throttling persists, and stop after a bounded number of attempts so remaining records can be retained for recovery.

Which implementation meets these requirements?

Show answerCORRECT ANSWER

Resubmit only UnprocessedItems. Before each attempt, wait for a randomly selected duration between zero and a capped delay ceiling that doubles after each unsuccessful attempt. Stop at the attempt limit and retain any remaining unprocessed records.

Retrying only UnprocessedItems avoids repeating accepted writes. The increasing delay ceiling reduces retry frequency during persistent throttling, and randomization spreads retries across workers. The attempt limit bounds the retry effort and allows unresolved records to be retained for recovery.

SAP-C03

A company uses an AWS Step Functions Standard Workflow to coordinate automated recovery of a shared application. A Task state requests a recovery lease from an internal service before any infrastructure changes occur. The Task makes one request per invocation and has no internal retries.

The operations team requires the following behavior:
• Remediation.Obsolete indicates that the incident has already been resolved. This error must go directly to the existing escalation handler without a retry.
• Remediation.LockBusy indicates that another recovery operation temporarily holds the lease. If this error persists, allow at most four retries, with configured delays of 1, 2, 4, and 8 seconds before the successive retries.
• Any other non-timeout Task error must receive at most two retries, with configured delays of 5 and 10 seconds.
• Timeouts must not be retried. After an applicable retry policy is exhausted, the error must go to the existing escalation handler.

The Task already has a Catch entry for States.ALL that routes errors to the escalation handler. No retry jitter is required.

Which Retry configuration should a solutions architect add to the Task state?

Show answerCORRECT ANSWER

"Retry": [ {"ErrorEquals": ["Remediation.Obsolete"], "MaxAttempts": 0}, {"ErrorEquals": ["Remediation.LockBusy"], "BackoffRate": 2, "MaxAttempts": 4}, {"ErrorEquals": ["States.TaskFailed"], "IntervalSeconds": 5, "BackoffRate": 2, "MaxAttempts": 2} ]

The first retrier prevents retries for Remediation.Obsolete. The LockBusy retrier defaults IntervalSeconds to 1; BackoffRate 2 and MaxAttempts 4 produce delays of 1, 2, 4, and 8 seconds. The final retrier handles other non-timeout Task errors with two retries after 5 and 10 seconds. States.TaskFailed excludes States.Timeout. Once a matching retrier is exhausted, Step Functions proceeds to Catch rather than falling through to another retrier.

DVA-C02

An AWS Lambda function is invoked synchronously to retrieve shipping estimates from a third-party HTTPS API. The requests are read-only. During traffic bursts, the API returns HTTP 429 responses without a Retry-After header. The function uses an HTTP client that has no automatic retries and currently retries every 100 milliseconds. Logs show that many concurrent invocations retry together, prolonging the throttling.

The developer must change the retry logic so that repeated throttling progressively reduces retry pressure and concurrent invocations do not produce synchronized retry spikes. Each invocation must make no more than five total attempts and stop retrying before its remaining execution time is exhausted.

Which implementation meets these requirements?

Show answerCORRECT ANSWER

After each HTTP 429 response, choose a new random delay between zero and min(2 seconds, 100 milliseconds × 2^(retry number − 1)) before retrying. Enforce the attempt limit and check the remaining execution time before waiting and making another request.

Capped exponential backoff increases the waiting window after consecutive throttling responses, reducing retry pressure. Full jitter independently randomizes each delay to spread concurrent retries. The attempt limit and remaining-time checks keep retries bounded within the invocation's execution budget.

Practice Exponential Backoff in a full exam

Questions on Exponential Backoff in each practice exam, with every answer option and an explanation for each:

Start a free practice exam

What is Exponential Backoff

4 min read

Exponential backoff controls when a client retries a failed operation. A typical schedule doubles the delay after each unsuccessful attempt: 200 milliseconds, 400 milliseconds, 800 milliseconds, and so on, up to a configured maximum. More generally, the delay before retry n is min(maximum delay, initial delay × multiplier^(n−1)), where n starts at 1. The multiplier is commonly 2, but doubling is not mandatory.

Randomization, called jitter, is essential when many clients share a dependency. Without jitter, clients that fail together can retry together, creating recurring traffic spikes. With full jitter, each retry waits for a random duration between zero and the exponentially increasing delay ceiling. The exponential schedule reduces retry frequency, while jitter spreads attempts across time.

Retry only failures that are plausibly temporary, such as throttling responses, selected server errors, connection failures, or certain timeouts. Authentication failures, malformed requests, and unsupported parameters generally require correction rather than another attempt. Follow the API's documented retry rules and honor server retry guidance, such as Retry-After, when supplied. A timeout is ambiguous: the server might have completed the operation even though the client never received the response.

For this reason, retries must be safe. Use idempotency tokens, conditional writes, deduplication keys, or transactional controls where appropriate. For example, retrying an ingestion operation after a lost response must not create duplicate business events or apply the same payment twice. Exponential backoff changes retry timing; it does not provide exactly-once execution or make a non-idempotent operation safe.

Cloud SDKs commonly implement exponential backoff for recognized retryable failures. Check the selected retry mode, maximum attempts, timeouts, and SDK-specific behavior before adding application retries. Some settings count the initial request as an attempt, whereas others count only retries. Nested retry policies can multiply downstream requests: an SDK, a Lambda function, and a workflow might each repeat the same operation. Allocate an overall retry budget and deadline across these layers.

In data ingestion, inspect partial-success responses rather than repeating an entire batch. DynamoDB BatchWriteItem can return UnprocessedItems in a successful response; retry those items with backoff. Kinesis Data Streams PutRecords can report failures for individual records; retry the failed entries and account for possible duplicates and ordering changes. Persistent throttling requires changes such as lower producer concurrency, better partition-key distribution, or additional capacity, not endlessly longer retries.

For Amazon RDS, distinguish AWS control-plane API throttling from database-engine errors. SDK retry policies for an RDS API request do not automatically retry SQL statements. Database reconnects, deadlocks, and serialization failures need driver- or application-specific handling, often including retrying an entire transaction. In AWS Glue or Amazon EMR pipelines, investigate throttling metrics, executor concurrency, hot partitions, and downstream connection limits before increasing job retries. Repeating a whole job can duplicate output and cost far more than retrying a small failed operation.

For automation, AWS Step Functions can apply configurable retry intervals, a backoff rate, maximum retries, and, where supported, a maximum delay and full jitter. Retry only appropriate task errors and route exhausted retries to recovery handling. Lambda code calling AWS APIs should use bounded SDK retries that fit within the function's remaining execution time. A long wait inside a function consumes execution duration; durable workflow waits or queued processing can be more appropriate for extended recovery.

Foundation-model integrations require both request-rate and token-aware controls. Retry eligible throttling responses or transient model failures within the application's latency budget, but do not retry an oversized prompt unchanged. A model timeout may still incur processing costs, and restarting after streaming output has begun can duplicate or contradict content already delivered. Define an explicit stream-recovery policy, bound attempts, and consider a compatible fallback model or graceful failure. API Gateway can enforce request throttling, but token budgets generally require application-level accounting.

Backoff is reactive retry control, not proactive rate limiting, extra capacity, or a quota increase. Combine it with concurrency limits, queues, admission control, and circuit breakers where appropriate. A standby environment must have sufficient regional quotas and usable capacity before failover; retries cannot compensate for missing capacity. Monitor attempt counts, accumulated retry delay, throttling, end-to-end latency, and final failures, using traces to distinguish original calls from retries. Bounded backoff reduces wasted requests and network traffic compared with immediate retry loops, but excessive retries still increase latency, compute duration, and potentially model or data-processing charges.

Learn Exponential Backoff in lessons

10 short lessons on Exponential Backoff in 5 certification courses.

and 4 more lessons

Open the lessons