Graceful Degradation

Graceful degradation starts by distinguishing essential outcomes from optional capabilities. An application should define what it can safely deliver when a dependency becomes slow, unavailable, or capacity-constrained. For example, a customer-support application might normally generate an answer from retrieved documentation, but during a model outage return relevant documentation links and offer escalation. The application remains useful without pretending that its full capability is available.

Design explicit degradation levels rather than improvising after a failure. A generative AI request might first use the preferred foundation model, then an approved alternative model, and finally a deterministic response or search-only experience. Each level needs an execution deadline, eligibility rules, capacity limits, and a clear response contract. A smaller model is not automatically an acceptable substitute: evaluate its accuracy, context window, tool support, structured-output behavior, safety controls, and handling of the application's languages and tasks.

Use timeouts and bounded retries before selecting a fallback. Configure AWS SDK retry behavior for exponential backoff with jitter where supported, and keep retries within the end-to-end latency budget. Retry transient failures such as throttling or selected server errors, rather than repeatedly submitting invalid requests. Coordinate SDK retries with workflow retries so nested retry policies do not multiply traffic. For workflows that invoke tools or perform writes, use idempotency controls because a timeout does not prove that the original action failed.

A circuit breaker prevents repeated calls to a dependency that is persistently failing. After a threshold is reached, requests bypass that dependency and use a fallback; limited probe requests later determine whether normal operation can resume. AWS Step Functions can orchestrate this behavior using tasks, choices, retries, catches, and shared state where necessary. Retry and Catch alone are not a complete circuit breaker, and breaker state must be shared when protection is intended to apply across workflow executions.

Regional resilience requires understanding the failure scope. Amazon Bedrock Cross-Region Inference uses inference profiles to route supported model requests among eligible Regions, helping address regional capacity constraints and supported regional availability differences. It does not provide arbitrary model substitution or guarantee recovery from every service disruption. Verify model support, source and destination Regions, quotas, IAM permissions, organization policies, and data-processing requirements. Separately deployed models in multiple Regions require application routing, sufficient standby capacity, synchronized artifacts and configuration, and tested failover procedures.

Protect fallback paths from overload. Amazon API Gateway throttling can limit incoming request rates, while application-level admission controls can prioritize essential requests and disable expensive optional features. A fallback that depends on the same exhausted quota or failed downstream system may provide no real isolation. Cached responses must respect authorization boundaries, freshness requirements, and tenant separation; a generic unavailable message is safer than exposing another user's data.

Degradation must not weaken security, compliance, or factual integrity. If retrieval fails, generating an unsupported answer may be worse than returning a clear limitation. If identity verification, authorization, or a mandatory safety check is unavailable, reject the affected operation rather than bypassing it. Explain material limitations to callers and distinguish reduced-service responses from normal successes in telemetry.

Measure fallback frequency, time spent degraded, latency, errors, cost, and output quality. AWS X-Ray traces can expose dependency failures and retry delays across instrumented service boundaries, while operational metrics and alarms reveal prolonged degradation. Test failure scenarios and controlled recovery so restored traffic does not immediately overload a recovering dependency. Graceful degradation complements redundancy and failover; unlike transparent failover, it explicitly permits reduced capability, and unlike unlimited retries, it bounds the work spent trying to recover each request.