Cloud concept · Updated

At-Least-Once Delivery - At-least-once delivery is a messaging guarantee that allows an accepted event to be delivered more than once so that recoverable failures do not silently prevent delivery.

In short

At-least-once delivery uses retries, acknowledgments, and durable buffering to favor delivery reliability over duplicate avoidance. It is common in real-time log ingestion, where consumers must tolerate repeated records without inflating metrics, duplicating alerts, or corrupting downstream state.

Cloud Concepts Hub →

6 key aspects of At-Least-Once Delivery

At-least-once delivery combines durable buffering and retries with consumer-side duplicate handling, bounded recovery policies, and explicit monitoring.

01
Acknowledgment-driven redelivery

Events remain eligible for redelivery until the relevant acknowledgment or checkpoint is recorded. A failure after output is committed but before acknowledgment is a normal source of duplicates.

02
Durability within defined limits

Durable buffers allow delivery to resume after temporary failures. Retention periods, retry policies, and terminal error handling still bound what the system can recover.

03
Duplicate-safe consumers

Use stable event identifiers, idempotent writes, or transactional deduplication to prevent repeated deliveries from producing repeated effects. Counter increments and external notifications require particular care.

04
Independent delivery boundaries

Evaluate producer submission, broker delivery, consumer processing, and destination writes separately. A guarantee at one boundary does not automatically extend across the entire pipeline.

05
Replay and operational recovery

Monitor backlog age, failures, and consumer lag, and retain events long enough to cover expected outages. Ensure replay uses the same identifiers and duplicate-handling rules as normal ingestion.

06
No implicit ordering guarantee

Duplicates, delayed retries, and concurrent consumers can affect observed order. Select and configure ordering mechanisms separately when processing depends on event sequence.

Cloud services for At-Least-Once Delivery

CloudWatch Logs subscriptions can forward incoming logs to Kinesis Data Streams, where retained records support buffering and replay. Lambda can consume the stream, transform records, and write them to downstream storage using stable event identifiers for duplicate-safe writes. In architectures that need a separate work queue, SQS Standard provides another at-least-once delivery boundary for downstream tasks. Each integration has its own acknowledgment, retry, and retention behavior. Monitor subscription delivery failures, stream consumer lag, function errors, and queue age together; successful acceptance by one service does not prove that the final destination committed the event.

Amazon Kinesis Data StreamsAmazon Kinesis Data Streams

Producer retries and consumer restarts can cause records to be written or processed more than once. Preserve application-level event identifiers, checkpoint appropriately, and size stream retention to support recovery and replay.

AWS LambdaAWS Lambda

Lambda event source mappings for streams and queues process events at least once, so a function can receive records more than once. Make destination writes idempotent and use supported partial-batch failure handling to reduce unnecessary retries of successful records.

At-Least-Once Delivery exam questions

Real practice exam questions on At-Least-Once Delivery, with the correct answer and its explanation. Answer them with every option in the exam simulator.

DEA-C01

A company is validating a streaming ingestion pipeline before production deployment. An AWS Lambda function processes records from Amazon Kinesis Data Streams through one event source mapping and adds each record's byte count to a usage report. Producers write each event only once.

During testing, the engineer finds that a record with the same stream ARN, shard ID, and sequence number appeared in two successful Lambda invocations. The usage report counted its bytes twice. The team proposes changing the event source mapping settings so that the reporting code never needs to handle repeated records.

Which assessment should the data engineer provide?

Show answerCORRECT ANSWER

Repeated processing is possible even when invocations succeed. The usage calculation must tolerate repeated records because event source mapping settings cannot guarantee single processing.

Lambda event source mappings for Kinesis process events at least once, and duplicate processing can occur. A successful invocation does not establish an exactly-once guarantee. The reporting design must ensure that processing a record again does not inflate usage totals.

SAP-C03

A marketplace is redesigning its product-catalog ingestion pipeline. Independent publishers submit complete product snapshots with an authoritative, monotonically increasing version number for each product. Network delays and publisher retries can cause older snapshots to reach ingestion after newer snapshots. Traffic is highly bursty, and snapshots for different products must be processed concurrently without provisioning messaging capacity.

The ingestion service must acknowledge a snapshot only after the messaging layer durably accepts it. Workers can be unavailable for up to 12 hours. After workers recover and drain the backlog, the catalog database must contain the highest accepted version of each product. Intermediate versions do not need to be applied, and duplicate deliveries must not change the final result. The database supports atomic conditional replacement of a product record based on its stored version.

Which solution meets these requirements?

Show answerCORRECT ANSWER

Use an Amazon SQS standard queue with a 4-day retention period. Inspect individual SendMessageBatch results, acknowledge only successful entries, and retry failed entries. Have workers atomically replace a product record only if the incoming version exceeds the stored version. Treat equal or older versions as successful no-ops, and delete messages after the database operation succeeds.

Standard queues support highly bursty traffic with nearly unlimited API-call throughput and durably buffer accepted messages independently of worker availability. Per-entry result handling prevents premature acknowledgment. Because delivery is at least once and ordering is best effort, the atomic version condition is essential: duplicates and stale snapshots become harmless no-ops, and a retry after a database commit cannot regress the catalog. Four-day retention covers the stated worker outage.

DVA-C02

An application receives events through an HTTPS endpoint subscribed to an Amazon SNS standard topic. The endpoint currently returns HTTP 200 immediately and then writes the event to an existing durable inbox. A separate worker processes inbox records. During endpoint restarts, some events never reach the inbox, even though SNS reports successful delivery.

Inbox writes normally complete within 100 milliseconds but occasionally fail because of temporary database unavailability. The inbox already handles duplicate events safely. The developer must prevent the endpoint from acknowledging events that have not been durably stored and must allow SNS to retry temporary write failures.

Which endpoint implementation meets these requirements?

Show answerCORRECT ANSWER

Write the event to the durable inbox before responding. Return HTTP 200 only after successful storage, and return HTTP 503 if temporary database unavailability prevents storage.

This places the acknowledgment after durable acceptance. HTTP 503 allows SNS to retry according to the subscription's delivery policy when storage is temporarily unavailable. If a successful write is followed by a lost response, SNS might deliver the event again, which the inbox can safely handle.

Practice At-Least-Once Delivery in a full exam

Questions on At-Least-Once Delivery in each practice exam, with every answer option and an explanation for each:

Start a free practice exam

What is At-Least-Once Delivery

3 min read

At-least-once delivery means that a messaging system can deliver the same accepted event multiple times while attempting to ensure delivery despite recoverable failures. The guarantee applies within a defined boundary, such as delivery from a broker to a consumer, and depends on the system's retention, retry, and failure-handling rules. It does not promise unlimited retries, permanent storage, or successful completion of downstream business operations.

The mechanism usually combines durable storage with acknowledgments or checkpoints. A producer submits an event, the transport retains it, and a consumer acknowledges it after completing the required work. If an acknowledgment is missing, the system can deliver the event again. For example, a log processor might successfully write a record to a search index and then crash before acknowledging the message. Redelivery is necessary because the transport cannot determine whether the write succeeded.

Duplicates can also originate at the producer. If a log shipper sends a batch but loses the response, it cannot distinguish a rejected request from an accepted request whose acknowledgment was lost. Retrying protects against missing logs but may insert the same events again. Batch APIs add another consideration: callers should inspect individual record results and retry failed records rather than automatically resubmitting every successful record.

In cloud log pipelines, subscriptions, streams, queues, and processing functions often introduce separate delivery boundaries. A log collection service might forward events into a stream, and a function might transform those events before writing them to storage. Reliability must be evaluated at every boundary. A downstream at-least-once transport cannot recover a log event that an upstream agent discarded before durable acceptance.

Consumers should therefore make repeated processing safe. Assign a stable event identifier before retries occur and preserve it throughout the pipeline. A search-index consumer can use that identifier as the document key so repeated writes replace the same document rather than create new documents. A metric aggregator needs stronger protection: blindly incrementing a counter for every delivery overcounts events. It can use a transactional deduplication record together with the counter update, or another design that makes the combined operation idempotent.

Deduplication itself has trade-offs. Its storage and lookup costs grow with event volume and the time window during which duplicates can arrive. Expiring deduplication entries too early allows old replays to affect results again. Hashing only the log text is usually insufficient because distinct events can legitimately contain identical messages. Separately recording an event as processed before committing its output can also cause loss if the consumer crashes between those actions.

Operational controls are equally important. Monitor ingestion failures, throttling, backlog age, consumer lag, retry volume, and dead-letter queues where supported. Configure sufficient retention and processing capacity to survive expected outages, and provide a replay procedure for failed records. Retries increase latency, request costs, and downstream load; backoff and capacity management help prevent a recovering destination from being overwhelmed.

For architecture and exam questions, distinguish delivery from processing and ordering. At-least-once delivery permits duplicates but does not inherently guarantee order or exactly-once effects. Real-time ingestion also does not mean instantaneous ingestion: retries and buffering can delay events. When missing logs is less acceptable than repeated logs, choose a reliable buffered path and pair it with duplicate-safe consumers rather than assuming the transport eliminates duplicates.

Learn At-Least-Once Delivery in lessons

5 short lessons on At-Least-Once Delivery in 3 certification courses.

Open the lessons