Rollback

Rollback is a recovery action that reverses a recent change rather than introducing a new fix. Its objective is to restore acceptable service quickly, not necessarily to reconstruct every detail of the previous environment. Engineers should define the rollback target explicitly: an application release, container image digest, function version, infrastructure template, configuration revision, or combination of these. A previous version is useful only if it remains compatible with the current data, dependencies, and security requirements.

Deployment methodology determines how rollback works. For an in-place deployment on virtual machines, recovery might redeploy the previous application artifact and configuration, requiring restarts and temporarily reduced capacity. A blue/green deployment can return traffic to the previous environment if that environment remains available and healthy. A canary deployment limits initial exposure and can route traffic away from the new version when validation fails. Rolling deployments replace instances or tasks incrementally; reversing them also takes time and can leave old and new versions running together during recovery.

Container and serverless platforms require platform-specific actions. An Amazon ECS service can use deployment failure detection with rollback enabled to return to its last completed deployment. In Amazon EKS, a Kubernetes Deployment can revert its Pod template to a retained revision, but a stalled rollout does not automatically undo itself by default. For AWS Lambda, an alias can be redirected to a previously published function version. These actions do not automatically reverse changes to external databases, queues, secrets, or other dependencies.

Configuration changes deserve the same controls as code releases. Suppose an event-driven automation changes a connection limit and application errors increase. A controlled recovery workflow can restore the recorded prior value, verify that the setting took effect, and confirm that error rates and latency recovered. Scope the change to affected resources, use least-privilege permissions, and make recovery steps safe to retry. Coordinate automation with infrastructure-as-code ownership so that the next reconciliation does not immediately reapply the defective configuration.

Automatic rollback should use meaningful signals such as deployment failures, failed synthetic transactions, elevated application error rates, or unacceptable latency. A process starting successfully is not proof that the release works. Define alarm evaluation periods and post-deployment observation time, including how missing metrics are handled. Overly sensitive alarms can reverse healthy releases, while weak health checks allow defective releases to proceed. Manual approval may be appropriate when reversal has substantial business or data consequences.

Rollback is not a database backup restore and is not a universal transaction spanning cloud resources. An application deployment may write records that the old version cannot interpret, send messages, or trigger external payments. Those effects usually require separate recovery or compensating actions. Prefer backward-compatible schema changes and expand-and-contract migrations, delaying destructive changes until older application versions are no longer needed. Restoring a database backup can discard valid writes made after the backup and therefore requires a separate data-recovery decision.

When troubleshooting a failed deployment, correlate release identifiers with logs, metrics, audit events, health-check results, permissions, quotas, and dependency changes. A failed new release may reveal an underlying capacity or networking problem that the previous release will also encounter. Preserve evidence before replacing resources where practical, stop further rollout, execute the recovery procedure, and validate both user-facing behavior and background processing. Infrastructure rollback can itself fail because resources were modified or deleted outside the deployment system.

Retaining old environments and artifacts increases cost, while shorter retention reduces recovery options. Rehearse rollback under realistic load and record recovery duration. If reversal is unsafe, impossible, or would reintroduce a security vulnerability, a tested forward fix may be preferable. A successful rollback restores service; it does not remove the need to investigate and correct the original defect.