Timeouts

A timeout places a limit on how long an application will wait for work to complete. Distributed applications need these limits because a dependency can remain reachable while responding too slowly to be useful. Without timeouts, stalled requests can occupy threads, connections, memory, and concurrency slots until an otherwise healthy application becomes unavailable. A timeout bounds waiting; it does not prove that the dependency failed or that the requested work never happened.

Configure timeouts at the application client as well as at infrastructure boundaries. A connection timeout limits connection establishment, while a read timeout commonly limits how long a client waits for incoming data. An overall request timeout or deadline limits the complete operation. Exact semantics depend on the client library: a read timeout may reset when more data arrives, and some settings do not cover DNS resolution, TLS negotiation, or waiting for a pooled connection. Verify which phases each setting includes rather than assuming one timeout bounds everything.

Choose limits from the caller's latency requirements and the dependency's measured latency distribution. Consider tail latency, connection setup, expected payload size, and the acceptable rate of prematurely abandoned requests. Very short limits can turn ordinary latency variation into errors and additional retries. Very long limits retain resources and delay failure responses. Increasing a timeout does not fix an overloaded dependency and can worsen resource exhaustion under sustained traffic.

Build a shared time budget across the request path. For example, an API with a three-second application deadline might allocate up to 800 milliseconds to each of two attempts against a third-party inventory API, with a short randomized backoff between attempts. The remaining budget must cover local processing, other dependencies, and returning the response. Before retrying, check whether enough time remains for another useful attempt. Where supported, propagate deadlines or remaining budgets so downstream calls cannot independently consume the caller's entire allowance.

Retry timeout errors only when the operation and available budget make retrying safe. Use a bounded number of attempts with exponential backoff and jitter, and avoid independent retry loops at multiple layers that multiply traffic. A payment provider might accept a charge even though its response never reaches the caller. Retrying that request without an idempotency key or another duplicate-prevention mechanism can create a second charge. Treat the result as uncertain and use the provider's idempotency or status-checking facilities.

Timeouts and circuit breakers solve different problems. A timeout limits one wait; a circuit breaker can stop new requests after repeated failures and later allow limited probes for recovery. Handle timeout exceptions explicitly and choose a response appropriate to the business operation, such as returning a controlled error, serving acceptable cached data, or moving suitable work to an asynchronous workflow. Do not disguise a failed mandatory operation as success.

Finally, cancellation is not guaranteed to stop remote work. An HTTP client abandoning a request or a gateway returning a timeout may leave the backend processing it. Release local resources, propagate cancellation when supported, and make side effects safe despite uncertain outcomes. Record timeout counts, dependency latency, retry attempts, and remaining deadlines, then test slow connections, stalled responses, and exhausted budgets to confirm that failure remains bounded.