Concurrency

Concurrency describes overlapping work: multiple requests, jobs, or operations are in progress before all of them have finished. A web application might handle another request while an earlier request waits for a database response. Both requests are concurrent even if a single processor alternates between them. Parallelism is different: it means work actually executes at the same instant, usually on multiple processor cores or machines. A system can therefore be concurrent without executing tasks in parallel.

Concurrency is also different from request rate and throughput. Request rate describes arrivals per unit of time; throughput describes completions per unit of time. Concurrency describes how much work is in progress. For a stable system, average in-progress work is approximately arrival rate multiplied by average time spent within the measured boundary. For example, 100 requests per second with an average processing duration of 0.2 seconds require about 20 concurrent processing slots. If processing duration rises to one second at the same rate, the average requirement rises to 100. These averages support capacity estimates, but bursts and variable durations require additional headroom.

Cloud applications implement concurrency through asynchronous operations, thread pools, worker processes, containers, and multiple service instances. An asynchronous server can keep many requests in progress while waiting for network responses without assigning a dedicated thread to every request. However, asynchronous execution does not eliminate CPU, memory, connection, or downstream capacity limits. Increasing the number of workers helps only while the resources they depend on can support the additional work.

A practical concurrency limit controls how many operations may be active at once. Suppose an order-processing application uses a database connection pool with 40 connections. Allowing hundreds of workers to issue database operations simultaneously may increase waiting, timeouts, and retries rather than throughput. A bounded worker pool and a queue can absorb a temporary burst while keeping database demand within a safe range. Persistent arrivals above processing capacity will still create an increasing backlog, so buffering is not a substitute for sufficient capacity.

AWS Lambda provides an exam-relevant example: concurrency measures function invocations being processed at the same time. Reserved concurrency sets aside part of the account's Regional concurrency allocation for a function and also caps that function's concurrency; it does not preinitialize execution environments. Provisioned concurrency maintains initialized environments to reduce startup latency, but it is not itself a maximum concurrency setting. Requests beyond provisioned capacity can use additional available concurrency unless another limit prevents them. Throttling behavior and retry handling depend on the invocation source.

Concurrency also introduces correctness risks. Two workers updating the same inventory record can overwrite each other's changes unless the application uses appropriate transactions, conditional writes, or other coordination. Retries can repeat side effects, so operations such as payment submission should use idempotency controls. During troubleshooting, correlate active work with duration, throughput, queue age, throttles, resource utilization, and downstream errors. Tune concurrency using load tests and measured bottlenecks rather than assuming that more simultaneous work always makes an application faster.