Cloud concept · Updated

Horizontal Scaling - Horizontal scaling adjusts capacity by adding or removing servers, containers, or other processing units that share a workload.

In short

Instead of making one machine larger, horizontal scaling distributes work across multiple resources. It helps applications handle changing demand and can improve resilience, while requiring suitable workload distribution and state management. When automated to match demand, it provides elasticity.

Cloud Concepts Hub →

6 key aspects of Horizontal Scaling

Horizontal scaling combines adjustable resource counts, parallel workload distribution, deliberate state management, and safe resource lifecycle controls.

01
Scale out and scale in

Increase capacity by adding processing units and reduce it by removing them. This differs from vertical scaling, which changes the capacity of an individual unit.

02
Elastic capacity adjustment

Auto scaling adjusts resource counts using demand signals or schedules within configured limits. Choose metrics that reflect workload pressure rather than assuming CPU utilization is always sufficient.

03
Workload distribution

Requests can be distributed through a load balancer, while asynchronous jobs can be distributed through a queue. Additional resources help only when useful work can run concurrently.

04
State management

Keep durable application data outside disposable processing instances where practical. Stateful workloads require deliberate replication, partitioning, or coordination strategies.

05
Safe resource lifecycle

Allow new resources to initialize before receiving traffic, and let departing resources drain connections or finish work. Account for startup delays when choosing scaling thresholds and minimum capacity.

06
Bottlenecks and failure domains

Test shared dependencies because adding application instances cannot remove every performance limit. Spread resources across appropriate failure domains when resilience is also required.

Cloud services for Horizontal Scaling

For a virtual-machine application, Amazon CloudWatch supplies demand measurements, Amazon EC2 Auto Scaling adjusts instance counts, and Elastic Load Balancing routes requests to healthy instances. Together, these services support elastic horizontal scaling, provided the application and its dependencies can handle parallel work.

For containers, Amazon ECS can adjust service task counts through ECS Service Auto Scaling; EC2-backed clusters also need sufficient host capacity. AWS Lambda manages execution-environment scaling without requiring engineers to manage a server fleet. Across these approaches, engineers must still configure limits, protect downstream dependencies, and monitor latency, errors, and resource utilization.

Amazon EC2 Auto ScalingAmazon EC2 Auto Scaling

Changes the number of EC2 instances in an Auto Scaling group through scaling policies, schedules, or manual desired-capacity changes. Automatic capacity adjustments provide elasticity within configured minimum and maximum limits.

AWS LambdaAWS Lambda

Scales managed execution environments to process concurrent function invocations, subject to concurrency limits and scaling behavior. Downstream systems still need enough capacity to handle the resulting parallel requests.

Amazon CloudWatchAmazon CloudWatch

Provides metrics and alarms that can inform or trigger scaling actions. Engineers can monitor resource utilization and publish application-specific metrics to better match capacity to workload pressure.

Horizontal Scaling exam questions

Real practice exam questions on Horizontal Scaling, with the correct answer and its explanation. Answer them with every option in the exam simulator.

MLA-C02 (beta)

A company uses an AWS Glue Spark job to ingest daily JSON exports from Amazon S3 into a training-data repository. The job reads thousands of files, calls coalesce(1), and then performs CPU-intensive validation independently on each record before writing the results. As data volume grows, the validation stage exceeds the ingestion deadline. Monitoring shows that only one executor processes validation tasks while the other executors remain idle.

The company must retain the existing worker type and run the ingestion as a single job run. The output can contain multiple files. Which change will allow validation throughput to increase as more workers are added?

Show answerCORRECT ANSWER

Replace coalesce(1) with repartition using enough partitions to keep the available executor cores busy, and increase the number of Glue workers as ingestion volume grows.

Repartitioning distributes records across multiple tasks so that executors can validate records concurrently. Adding workers of the same type then increases aggregate processing capacity through horizontal scaling.

SAP-C03

A company runs a stateless API on Amazon EKS. A Kubernetes Horizontal Pod Autoscaler (HPA) scales the API Deployment from 6 to 80 replicas by using CPU utilization. Load tests confirm that the metric and resource requests are appropriate. The cluster has sufficient node capacity for 80 replicas.

Traffic arrives in bursts separated by 2 to 5 minutes of low demand. The company needs rapid scale-out but wants scale-in decisions to retain the highest replica recommendation from the preceding 10 minutes. Currently, the HPA has no scale-down stabilization window. Pods are repeatedly removed between bursts and recreated during the next burst.

A GitOps controller also reconciles the Deployment every minute. Its manifest specifies replicas: 6, and audit logs show that the controller repeatedly resets the replica count after the HPA increases it. The company must continue using GitOps to manage application images and other Deployment settings.

Which solution meets these requirements?

Show answerCORRECT ANSWER

Configure GitOps to stop managing the Deployment's spec.replicas field and leave replica control to the HPA. Set the HPA scaleDown stabilizationWindowSeconds to 600 and scaleUp stabilizationWindowSeconds to 0, with scale-up policies that permit rapid expansion.

Removing GitOps ownership of spec.replicas prevents reconciliation from undoing HPA scaling decisions while preserving management of other Deployment fields. A 600-second scale-down stabilization window uses the highest desired replica recommendation from that window to limit scale-in. A zero scale-up stabilization window and suitable scale-up policies allow rapid expansion.

SOA-C03

A company runs a read-heavy application on a provisioned Amazon Aurora MySQL cluster with one writer and two Aurora Replicas. Read-only connections use the reader endpoint, and connection pools regularly recycle connections. Traffic increases unpredictably.

Load testing shows that each reader meets the application's latency requirement with up to 100 concurrent database connections. Above that level, latency increases even though reader CPU utilization remains below 30%. Almost all connections are actively executing queries, and the database connection limit is not being reached. The writer has ample capacity.

A CloudOps engineer must automatically add and remove readers as connection demand changes, maintaining between two and eight readers. Which configuration meets these requirements?

Show answerCORRECT ANSWER

Configure Aurora Auto Scaling with two to eight replicas and a target tracking policy that targets 100 average database connections per reader.

Aurora Auto Scaling supports target tracking with the predefined RDSReaderAverageDatabaseConnections metric. A target of 100 aligns scaling with the tested per-reader workload limit. The policy can add readers as average connection demand increases and remove them as demand falls, within the configured limits.

Practice Horizontal Scaling in a full exam

Questions on Horizontal Scaling in each practice exam, with every answer option and an explanation for each:

Start a free practice exam

What is Horizontal Scaling

2 min read

Horizontal scaling changes the number of resources serving a workload rather than changing the size of an individual resource. Adding resources is called scaling out; removing them is called scaling in. For example, a web application might expand from two application servers to six during a busy period, then return to two when traffic decreases. Vertical scaling, by contrast, changes the CPU, memory, or other capacity of an individual server.

In cloud environments, horizontal scaling commonly uses virtual machines, container tasks, or managed execution environments. A load balancer distributes incoming requests across available application instances. For background processing, multiple workers can consume jobs from a shared queue without requiring a request-routing load balancer. Each additional resource contributes useful capacity only when the workload can be divided among resources.

Auto scaling provides elasticity by automatically adjusting capacity to changing demand. Scaling policies can respond to measurements such as CPU utilization, requests per target, or queue backlog per worker. Scheduled scaling can prepare capacity for predictable traffic patterns. In AWS, Amazon EC2 Auto Scaling can change the number of EC2 instances in an Auto Scaling group within configured limits. For certification questions, distinguish scalability, the ability to handle changing workload, from elasticity, the adjustment of provisioned capacity to match demand.

Applications scale more easily when individual processing instances do not hold essential local state. A shopping application can store its cart data in a shared data store so that any application instance can handle the next request. Keeping sessions or uploaded files only on one instance creates routing dependencies and risks losing data when that instance is removed. Stateful systems can also scale horizontally, but may require replication, partitioning, and explicit consistency decisions.

Scaling is not instantaneous. New resources need time to start, initialize dependencies, and pass readiness checks before accepting work. Engineers should configure minimum and maximum capacity, appropriate scaling metrics, and instance warmup behavior. During scale-in, connection draining and graceful shutdown help avoid interrupting requests or abandoning jobs. Queue workers should tolerate retries and use idempotent processing where duplicate delivery is possible.

More application instances do not automatically produce proportional performance gains. A shared database, external API rate limit, network constraint, or serialized operation may remain the bottleneck. Horizontal scaling also adds costs and operational complexity, including monitoring, distributed tracing, and coordination. Scaling in can reduce usage-based charges, but not every associated cost disappears when compute resources are removed.

Horizontal scaling is not synonymous with automatic scaling or high availability. Engineers can add instances manually, and multiple instances can still share one failure domain. Distributing capacity across independent failure domains improves resilience, while auto scaling can restore or adjust capacity. A load balancer distributes traffic but does not, by itself, create additional application instances.

Learn Horizontal Scaling in lessons

13 short lessons on Horizontal Scaling in 6 certification courses.

and 7 more lessons

Open the lessons