Circuit Breaker

A circuit breaker protects a caller from repeatedly invoking an unhealthy dependency. Without one, slow or failing model endpoints, databases, or external tools can consume connection pools, execution capacity, and request budgets. Retries can amplify the disruption by adding more traffic to an already overloaded service. A breaker interrupts that feedback loop and gives the dependency time to recover.

The usual states are closed, open, and half-open. When closed, requests pass through and their outcomes are measured. If failures exceed a configured threshold, the breaker opens and subsequent calls fail immediately or follow a fallback path. After a cooldown, the half-open state permits a limited number of probes. Successful probes close the breaker; failed probes reopen it. An open breaker blocks calls, despite the everyday association of an open gate with allowing passage.

Define failures according to the dependency and operation. Timeouts, connection failures, selected server errors, and sustained throttling can be appropriate signals. Invalid requests and authorization errors usually require correction rather than a recovery cooldown. A practical policy might open after a high failure percentage within a rolling window, provided a minimum number of requests has occurred. Thresholds must reflect normal latency and traffic volume; an overly sensitive policy can unnecessarily disable a healthy dependency.

In cloud workflows, orchestration can check breaker state before invoking a dependency and route failures to a handler that updates that state. AWS Step Functions can implement this control flow with Choice, Task, Wait, and error-handling constructs, but Retry and Catch alone are not a shared circuit breaker. Separate workflow executions need shared state if failures across callers should open the same circuit. A store such as Amazon DynamoDB can maintain the state, cooldown deadline, and probe ownership using conditional updates. Key the state by the intended failure boundary, such as model, endpoint, Region, and operation, rather than disabling unrelated dependencies.

For foundation model integration, opening a circuit around one endpoint can trigger an approved fallback model, a separately deployed regional endpoint, a cached response where appropriate, or an explicit temporary-unavailability response. Amazon Bedrock cross-Region inference uses supported inference profiles to route requests among supported Regions; it is not an application circuit breaker and does not make every model available everywhere. Cross-Region model deployments also require suitable capacity, configuration, and data access. Validate model compatibility, output quality, permissions, and data-residency requirements before enabling either fallback approach.

Agentic workflows need additional stopping conditions for maximum tool calls, elapsed time, token consumption, and repeated unsuccessful actions. A circuit breaker can isolate an unavailable tool, while explicit workflow checks stop unproductive agent loops. Lambda functions can enforce client request deadlines and have an execution timeout, but a timeout does not guarantee that a remote operation was canceled. Use idempotency for retried side-effecting actions. IAM policies independently restrict permitted resources and actions; neither a circuit breaker nor an agent stopping rule replaces authorization.

Combine the breaker with bounded retries, exponential backoff with jitter, request deadlines, and concurrency limits. Coordinate retry behavior across SDKs and workflow layers to avoid multiplying attempts. Trade-offs include shared-state overhead, inconsistent observations across callers, false trips, and delayed recovery during cooldown. Monitor state changes, rejected calls, probe results, latency, and fallback use. Test both outage and recovery behavior, including concurrent half-open probes and failure of the breaker-state store itself.