15 min.

AWS compute services with appropriate use cases

A comprehensive overview of key compute services, including Amazon EC2, Amazon Fargate, AWS Lambda, and Lightsail, as well as supportive compute services like AWS Batch and Amazon EMR, among others

Compute is often considered a power that originates from the CPU and processes data. However, to run a program, more components are required, including at least Memory and Storage.

 

Compute realized on AWS is commonly referred to as 'Cloud Computing'. Simplified, it's computing available over a network on demand, with a default pay-as-you-go pricing model. These three attributes make cloud computing very appealing for anyone eager to implement any idea related to software development in the fastest possible Time To Market (TTM), at least in terms of the hardware part.

 

Compute is one of many offerings from AWS, including Databases, Analytics, Machine Learning, Networking, and more. However, computing is the most versatile of all categories because it can be utilized for almost anything. In other words, it enables the creation of highly customized solutions that can easily integrate with other AWS services. For example, databases can also be run using AWS compute services without RDS.

 

Below is a comprehensive list of compute services available on AWS, with a focus on their use cases.

 

 

Amazon EC2

 

By many, and likely by AWS itself, EC2 instances are considered the cornerstone of cloud computing on AWS. These Virtual Machines offer a broad array of configurations to cater to different computing needs. These configurations, which vary in CPU, memory, network, and EBS bandwidth, are identified by unique codes such as M7g or T3.

 

 

EC2 Key Features

 

Instances

 

EC2 are Virtual Machines with fixed hardware specifications, customizable in terms of OS, networking, security, and storage. They can perform virtually any task that a non-virtualized operating system can.

 

 

Amazon Machine Images (AMIs)

 

Pre-packaged software stacks, including operating systems and applications, for instance creation and customization. Users can also create their own AMIs from existing instances, supporting immutable infrastructure strategies.

 

 

Instance Types

 

Offer various balances of CPU, memory, storage, and networking capabilities to suit different applications.

 

 

Security Groups

 

Act as virtual firewalls to control instance traffic, enhancing network security.

 

 

Elastic Block Store (EBS) Volumes

 

Provide persistent block storage for data that can be attached to instances.

 

 

Instance Store

 

Offers faster, directly attached ephemeral storage, though data is lost if the instance is stopped or terminated.

 

 

Elastic IP Addresses

 

Static public IP addresses for dynamic cloud computing, ensuring accessible instances even through stop/start cycles.

 

 

Virtual Private Clouds (VPCs)

 

Enable launching of AWS resources in a customizable, isolated network, offering enhanced virtual networking control.

 

 

Auto Scaling

 

Dynamically adjusts instance capacity to maintain application performance according to demand.

 

 

Placement Groups

 

Allow instance clustering within an Availability Zone for high throughput and low latency, or distribution across zones for increased fault tolerance.

 

 

EC2 Use Cases

 

Instance Types

 

EC2 instances are organized into categories such as General Purpose or Storage Optimized, each designed for specific workloads and applications. These categories represent dedicated computing use cases.

 

While it's possible to use a specific instance type for various applications, AWS suggests deploying multiple instances in parallel and evaluating their performance to identify the optimal configuration. 

 

It's also worth noting that the use cases for some categories may intersect with others.

 

 

Instance TypeUse Cases
General PurposeIdeal for starters if you're unsure which instance type to use. They are the most balanced of all, including the cheapest instances. Possible use cases include small projects, development, and applications that demand similar resources for all components without stressing any particular one, like web servers or code repositories.

Compute Optimized

 

Best for workloads that are primarily CPU-intensive. Possible use cases are high-performance web servers, gaming servers, batch processing, and other compute-sensitive applications.
Memory OptimizedSuitable for memory-sensitive applications such as in-memory databases, in-memory caches, genomic and real-time big data analytics
Accelerated ComputingThese instances utilize hardware accelerators and co-processors, making them ideal for graphics processing, machine learning (ML) and deep learning computations, scientific and engineering workloads, cryptocurrency mining, real-time video streaming, and financial modeling involving Monte Carlo simulations.
Storage OptimizedStorage-optimized instances are tailored for applications that heavily utilize storage. Typical use cases include large databases, analytics with extensive data lakes, data warehousing, and backup and disaster recovery solutions.
HPC OptimizedIdeal for applications requiring high-performance computing (HPC) processors, such as scientific and engineering simulations, machine learning and deep learning training of complex models, and render farms.

 

 

Operating System

 

They support a wide range of operating systems, such as Linux, Windows, and macOS, broadening their application spectrum. 

 

 

Operating SystemUse Cases
LinuxAmazon Linux 2 is designed for a broad range of computing environments, from virtualized to containerized workloads. It is ideal for hosting databases, performing data analytics, running line-of-business and web applications, as well as desktop applications.
WindowsWindows Server on EC2 is tailored for deploying Windows-based environments in the cloud. It supports a wide array of Windows applications and services, including Active Directory, SQL Server, and Internet Information Services (IIS). These instances provide a robust platform for running enterprise applications, web servers, and other critical Windows workloads, ensuring seamless integration with existing Windows infrastructure.
macOSEC2 Mac instances offer a cloud-based solution for developers working on Apple's ecosystem, streamlining the development process for apps across Apple devices. With quick provisioning and access to macOS environments, developers can efficiently build, test, and deploy applications for iPhone, iPad, Apple Watch, and Mac. This capability accelerates the development cycle for iOS and macOS apps by providing scalable and flexible access to macOS resources on AWS

 

 

Pricing Models

 

EC2 provides multiple pricing options, facilitating cost-effective deployment for a variety of needs.

 

Pricing ModelUse Cases
On-Demand InstancesIdeal for applications experiencing short-term fluctuations, irregular demand, or unpredictable workloads that require continuous operation. You're billed only for the seconds your resources are in use, without any long-term obligations
Spot InstancesIdeal for intermediate and applications with flexible scheduling where computing can be interrupted like HPC, batch processing jobs, BIig data.  Offers access to surplus Amazon EC2 computing capacity at discounts of up to 90% compared to On-Demand prices
Reserved InstancesIdeal for long term projects with predictable workload. Offers reserved capacity with a notable discount of up to 75% relative to On-Demand rates, making it an excellent choice for commitments of 1 or 3 years.

 

 

Tenancy

 

The choice of tenancy affects application deployment by dictating how instances are distributed across physical servers. While AWS documentation often links tenancy with the pricing model, its actual significance lies in the operational aspects of EC2 instance allocation. Therefore, a distinct categorization has been created to outline EC2 use cases focused explicitly on their tenancy options.

 

 

TenancyUse Cases
Shared TenancyBest for a wide range of general-purpose applications without strict compliance or isolation needs. Ideal for optimizing expenses, leveraging AWS's efficient resource allocation. Supports dynamic scaling, making it suitable for fluctuating demand.
Dedicated InstancesEssential for meeting legal standards that demand physical isolation from the hardware of other tenants. Offers an added layer of security for data-sensitive workloads by ensuring hardware dedication.
Dedicated HostsFacilitates compliance with core- or socket-based licensing terms, necessary for certain enterprise software. Provides granular control over server deployment and configuration, aligning with specific on-premises environments for seamless migration or hybrid setups. 

Allows for the application of pre-owned software licenses (Bring Your Own License - BYOL), reducing operational costs for software like Windows Server, SQL Server, and SUSE Linux Enterprise Server.

 

 

Placement groups

 

Placement groups in Amazon EC2 are a configuration that allows users to influence the placement of instances within the underlying hardware to optimize performance.

 

 

Placement GroupUse Cases
Cluster Placement GroupsHigh-Performance Computing (HPC): Ideal for workloads such as batch processing and media encoding, which demand high network throughput and low latency between instances. This setup benefits distributed applications, including distributed databases and analytics clusters, by providing very low latency communications.
Spread Placement Groups

High Availability Applications: Suitable for critical applications where the failure of a single instance or the compromise of availability zone availability could detrimentally affect the entire application. Spread placement groups enhance resilience by distributing instances across separate physical hardware, thereby reducing the risk of simultaneous failure.

 

Critical Instance Isolation: Applications that operate with a limited number of essential instances requiring isolation from each other, due to compliance or security considerations, gain from using spread placement groups. This ensures that instances do not share the same physical resources.

Partition Placement Groups

Large Distributed Systems: Optimized for extensive distributed and replicated systems such as Hadoop, Cassandra, and Kafka. Partition placement groups distribute instances across multiple hardware sets, minimizing the risk of correlated failures and enhancing system reliability.

 

Low Latency Distributed Workloads: Particularly advantageous for distributed file systems (e.g., HDFS), database systems (e.g., HBase), and other applications like Cassandra that necessitate very low latency interactions between instances. This configuration helps maintain performance and efficiency in large-scale distributed environments.

 

 

Amazon Lightsail

 

Amazon Lightsail is simillary to EC2 a Virtual Machine but compared to EC2 simpler to lauch and manage with less features but also cheapier.

 

 

Lightsail Key Features

 

Preconfigured Virtual Machines

 

Lightsail offers instance plans with preconfigured virtual machines, including various operating systems and software stacks like WordPress, LAMP (Linux, Apache, MySQL, PHP), and Node.js.

 

Fixed, Predictable Pricing

 

With monthly plans starting at $3.50, Lightsail provides a predictable pricing model that includes compute power, memory, storage, and data transfer, simplifying budgeting for users.

 

 

Simplified Resource Management

 

Users can easily manage instances, storage, databases, and more from a single dashboard, offering a straightforward approach to cloud resource administration. Lightsail offers managed databases that include automatic backups, patching, and encryption, simplifying database administration tasks.

 

Integrated Networking Features

 

Lightsail includes essential networking capabilities like static IP addresses, DNS management, and a managed load balancer, aiding in the seamless scaling of applications.

 

 

Snapshots

 

The service enables users to create snapshots of their instances and databases for backups or cloning, enhancing data protection and recovery strategies.

 

 

SSD-Based Storage

 

All plans come with SSD-based storage, ensuring high-performance solutions for application data storage needs.

 

 

AWS Integration

 

While designed for simplicity, Lightsail can connect with more advanced AWS services, allowing for application scaling as user needs grow.


 

 

Lightsail Use Cases

 

Simplicity and Accessibility

 

Ideal for individual developers, hobbyists, and students looking to grasp cloud computing basics.

 

Enables hosting of personal projects, websites, and web applications with ease.

 

 

Ideal for SMBs

 

Small and medium-sized businesses benefit from Lightsail's simplicity in running production workloads.

 

Offers straightforward management and cost-effectiveness.

 

 

Development and Testing

 

Organizations use Lightsail to create development and testing environments.

Facilitates rapid provisioning on demand, enhancing project agility.

 

 

Amazon EC2 vs Amazon Lightsail

 

 

FeatureAmazon EC2Amazon Lightsail
Control and FlexibilityHigh control over infrastructure with customizable instance types, operating systems, and storage options.Predefined instance bundles with fixed configurations for simpler setup.
Pricing ModelsPay-as-you-go based on actual usage with options for On-Demand, Reserved, and Spot Instances.Fixed monthly pricing plans starting at $3.50, making budgeting straightforward.
Resource ManagementManage resources (instances, storage volumes, etc.) separately for granular control.Single control panel for managing pre-configured resources like instances, storage, and load balancers.
Advanced FeaturesSupports dedicated hosts, container services, GPU instances, and more for complex applications.Focused on simpler workloads and quick deployments without the advanced features found in EC2.
Monitoring and ManagementDetailed monitoring and management with CloudWatch, CloudTrail, and more.Basic monitoring capabilities suitable for less complex needs.
ScalabilityFlexible scaling options allowing resources to be adjusted independently as needs grow.Fixed instance configurations and plans, with some limitations on scalability.

 

 

AWS Fargate

 

Unlike EC2 and Lightsail, AWS Fargate is serverless, making it distinct as it removes the need to manage servers. It serves as a compute engine for containers that can be integrated with multiple AWS services such as ECS, EKS, and AWS Batch, among others.

 

 

Fargate Key Features

 

Serverless Operation

 

Fargate manages the container runtime and underlying infrastructure, allowing you to focus solely on application development and deployment.

 

 

Resource-Based Pricing

 

Charges are based on the vCPU and memory resources your containers use, ensuring you only pay for what you consume.

 

 

Enhanced Security

 

Each task or pod operates within its own isolated environment, enhancing security by preventing resource sharing.

 

 

Flexible Orchestration

 

Fully compatible with Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS), Fargate offers versatile options for running containerized applications.

 

 

Effortless Scaling and Management

 

Fargate automates scaling, patching, and operational tasks, streamlining application management.

 

 

Integrated AWS Services

 

Includes native integration with AWS services such as CloudWatch for comprehensive metrics and logs, and supports AWS load balancers like ALB and NLB for efficient traffic distribution.

  •  

Fargate Use Cases

 

Optimizing Large and Small Workloads

 

Fargate is effective for both large workloads needing minimal overhead and small, bursty workloads, as it offers a pay-for-what-you-use model, making it cost-effective for variable demands.

 

 

Microservices and Serverless Applications

 

It simplifies deploying and scaling numerous microservices, allowing for easy management of large numbers of small services, enhancing agility and operational efficiency.

 

 

Batch and Event-driven Processing

 

Ideal for running batch jobs or handling event-driven tasks such as IoT data processing or real-time file processing, Fargate responds to events or schedules without the need for infrastructure provisioning.

 

 

Applications requiring isolated comoputing

 

Provides improved isolation and security for applications, as each task runs in its own security context without sharing resources with others.

 

 

CI/CD and Web Applications

 

Supports continuous integration and deployment pipelines, offering a consistent environment for testing and deploying applications. It's also well-suited for web applications and APIs requiring scalable, highly available backends.

 

 

Machine Learning and High Availability

 

Facilitates machine learning workloads by providing scalable compute resources and supports disaster recovery strategies by enabling quick deployment across multiple regions or availability zones.

 

 

AWS Lambda

 

AWS Lambda and AWS Fargate were briefly mentioned in the article "Serverless technologies and patterns on AWS," which explained that they are managed, predominantly serverless services focusing on code execution.
 

Lambda Key Features

 

Function Configuration

 

Customize memory, timeout settings, and permissions through the AWS console or CLI to tailor Lambda functions to your specific needs.

 

 

Environment Variables

 

Adjust function behavior dynamically without altering the code by utilizing environment variables.

 

 

Versioning

 

Implement version control on your functions to streamline the deployment process for testing or production environments.

 

 

Container Support

 

Deploy containerized applications using AWS-provided base images or custom ones, integrating seamlessly with Lambda's serverless architecture.

 

 

Layers

 

Optimize function deployment by separating common libraries and dependencies into layers, reducing overall function size.

 

 

Lambda Extensions

 

Enhance monitoring and security capabilities with extensions from AWS and third-party providers like Datadog and New Relic.

 

 

Function URLs

 

Create dedicated HTTP/HTTPS endpoints for direct function invocation over the internet, simplifying access.

 

 

Response Streaming

 

Enable streaming of large response payloads to boost performance and response times.

 

 

Concurrency Controls

 

Manage and automatically scale concurrent executions to maintain performance during demand spikes.

 

 

Lambda Use Cases

 

Serverless Applications

 

Enables the construction of fully serverless architectures that scale automatically, integrating seamlessly with AWS services like API Gateway and DynamoDB for diverse applications, including web and mobile backends.

 

 

Data Processing

 

Offers capabilities to efficiently process data in real time or in batches from sources such as Amazon S3, Kinesis, and DynamoDB streams. This supports tasks like ETL operations, analytics, and stream processing, facilitating file resizing, format conversion, and data transformation.

 

 

Web and Mobile Backends

 

Lambda serves as a dynamic backend solution for web and mobile applications, handling API requests, user authentication, and personalized content generation without server management overhead.

 

 

IoT Applications

 

Supports the development of serverless IoT application backends that process and respond to data from sensors and devices, enabling real-time analytics, automation, and efficient data management.

 

 

Scheduled and Event-Driven Tasks

 

Executes code in response to AWS service events or external triggers, and runs scheduled tasks using cron expressions, catering to a wide array of automated workflows and operational tasks.

 

 

AWS Outposts

 

AWS Outposts bring AWS infrastructure, services, and tools directly to on-premises locations, offering a comprehensive suite of features for a seamless hybrid cloud experience.

 

 

Outposts Key Features

 

AWS On-Premises Extension

 

Delivers AWS infrastructure and services on-premises, supporting applications with low latency, local data processing, and storage needs, enabling the use of EC2, ECS, EKS, and VPC networking locally.

 

 

Fully Managed and Maintained by AWS

 

Outposts equipment, including racks and servers available in 1U and 2U form factors equipped with Graviton2 and Intel Xeon processors, are fully managed, maintained, and updated by AWS, covering delivery, infrastructure maintenance, and software updates.

 

 

Seamless Service Integration

 

Integrates a wide array of AWS services on-premises, using familiar AWS APIs and tools for computing, storage, databases, and more, including specialized services like S3 on Outposts for local data compliance and latency reduction.

 

 

Unified Governance and Management

 

Facilitates unified access and resource management across cloud and on-premises environments with AWS governance services like Organizations, RAM, and CloudFormation.

 

 

Consistent Hybrid Cloud Experience

 

Ensures a consistent AWS experience across cloud and on-premises environments, simplifying application migration, hybrid application running, and connection to AWS's extensive service range.

 

 

Designed for Compliance and Security

 

Adheres to stringent compliance standards, providing robust security and data protection measures comparable to those in the AWS cloud.

 

 

Network Integration

 

Easily integrates with existing on-premises networks for a cohesive networking experience, connecting local resources with the AWS cloud efficiently.

 

 

VMware Cloud Support

 

Supports a fully managed VMware Cloud experience on-premises, allowing VMware workloads to run on Outpost's physical infrastructure with familiar VMware management tools.

 

 

Outposts Use Cases

 

Low-Latency Access

 

Essential for industries like financial trading, gaming, manufacturing, healthcare, and media, where immediate data processing and response times are crucial for performance and user experience.

 

 

Compliance and Data Residency

 

Ideal for healthcare, financial services, and other sectors with strict data residency and compliance requirements, allowing data to be stored and processed locally while leveraging AWS cloud services.

 

 

Efficient Data Processing

 

Enables processing large volumes of data generated on-premises, such as in manufacturing and industrial settings, reducing transfer costs and network usage before moving data to AWS for further analysis or storage.

 

 

Containerized Workloads

 

Supports running ECS or EKS clusters on-premises for applications that require integration with local systems, providing a seamless environment for container management.

 

 

Local Storage Solutions

 

Offers local storage options like S3 on Outposts, providing near real-time access to data with the familiar AWS interface, suitable for applications that demand immediate data availability.

 

 

Managed Infrastructure

 

Utilizes AWS's managed infrastructure and operational expertise for deploying and managing on-premises workloads, eliminating the need for extensive data center maintenance.

 

 

Hybrid Cloud and Edge Computing

 

Facilitates a smooth transition to the cloud for businesses and supports edge computing needs in telecommunications and media content delivery, ensuring applications are deployed closer to end-users or data sources.

 

 

Consistent Development Environment

 

Aids software development by mirroring AWS cloud configurations on-premises, ensuring a uniform environment across development, testing, and production stages.

 

 

Support for Highly Regulated Industries

 

Enables government and public sector entities to benefit from AWS services while adhering to regulatory requirements by keeping sensitive data within on-premises environments.

 

 

Other compute services

 


The following services play a crucial role in computing, leveraging computation based on EC2, Fargate, or on-premises compute power (AWS Outposts) and providing additional features for various scenarios but they are not independable compute services compares to the allready covered like EC2 or Fargate. Some of the following services were mentioned in “How to migrate applications into containers on AWS” article.

 

 

AWS Batch

 

AWS Batch differs significantly from previous computing services like EC2 or Lambda. Its primary focus is on executing specific jobs rather than hosting entire applications. In this context, a compute job is considered a distinct part of an application that runs autonomously, without any user interaction.

 

Batch Key Features

 

Comprehensive Management

 

AWS Batch automates the management of compute environments and job queues, facilitating the execution of a vast number of jobs of varying scales. It integrates seamlessly with Amazon ECS, Amazon EKS, and AWS Fargate, offering the choice between Spot and on-demand resources.

 

 

Automated Resource Provisioning

 

The service simplifies job submission by allowing you to define jobs and submit them to a queue. AWS Batch intelligently schedules these jobs and provisions additional AWS capacity as required, ensuring optimal resource utilization.

 

 

Monitoring and Scaling

 

AWS Batch monitors job progress, scaling down resources when they're no longer needed, thus ensuring efficient use of computing resources.

 

 

Workflow Integration

 

It supports complex workflows, allowing for the submission of jobs that are part of a larger pipeline. This feature enables the definition of dependencies between jobs, facilitating coordinated execution.

 

 

Custom Compute Strategies

 

With support for Amazon EC2 Launch Templates, AWS Batch enables customized compute resource templates, allowing for precise scaling based on job requirements. Additionally, flexible allocation strategies let users optimize for throughput and cost.

 

 

Cost and Resource Efficiency

 

Leveraging AWS's Spot Instances and its dynamic scaling capabilities, AWS Batch minimizes costs associated with batch computing. Payment is only required for the actual AWS resources utilized, enhancing cost optimization.

 

 

Consistent Execution Environment

 

By supporting containerization with Docker, AWS Batch ensures that jobs run consistently across different computing environments, streamlining development and deployment processes.

 

 

Batch Use Cases

 

High-Performance Computing

 

Facilitates a broad spectrum of compute-intensive tasks such as genomics analysis, financial risk modeling, deep learning, Monte Carlo simulations, and engineering simulations by efficiently running large numbers of parallel jobs.

 

 

Media Processing

 

Supports tasks like audio/video transcoding, image processing, and computer vision tasks, applying uniform operations across numerous files or objects to facilitate content delivery and analysis.

 

 

Scientific Research and Simulations

 

Empowers scientific and engineering disciplines with the capability to perform extensive simulations, including parameter studies or iterative simulations, crucial for achieving accurate results in research and development projects.

 

 

Software Development and Testing

 

Enhances the build and test cycles of distributed applications, particularly those utilizing container technologies like Docker, by automating and scaling test executions.

 

 

Data Processing and ETL

 

Optimizes data transformation processes, including ETL jobs for analytics, data warehousing, and report generation, handling vast data volumes from various sources for insightful analytics and decision-making.

 

 

Machine Learning Workflows

 

Streamlines the training and deployment of machine learning models, handling large datasets with optimized resource allocation to accelerate development and application of AI technologies.

 

 

Financial Analysis

 

Enables detailed financial analysis and modeling, including risk assessment and fraud detection, by processing transactional data in batches for comprehensive analysis and insights.

 

 

AWS App Runner

 

App Runner fully abstracts the resources required to run an application, requiring developers to only supply a code repository or a container image. The specific compute resources used by App Runner under the hood are not explicitly stated, leading to speculation that it might utilize EC2, Fargate, or both.

 

App Runner Key Features

 

Serverless Deployment

 

Enables the deployment of containerized applications directly from source code or container images, supporting a variety of programming languages and frameworks such as Node.js, Python, PHP, Ruby, and .NET Core, without the need for Docker expertise or managing servers.

 

 

Automatic Scaling and Management

 

Seamlessly scales applications to match demand and manages infrastructure tasks like provisioning, security updates, and maintenance, allowing developers to concentrate on their code.

 

 

Comprehensive Integration

 

Offers integration with AWS services like Lambda, API Gateway, Route 53, and X-Ray, enhancing application functionality. It also supports CI/CD through GitHub/CodeCommit, facilitating continuous code deployments.

 

 

Built-in Security and Compliance

 

Provides encryption in transit and integrates with AWS IAM for access management, ensuring a secure environment for applications.

 

 

Custom Domains and HTTPS

 

Supports the use of custom domains with AWS-managed SSL certificates for HTTPS, enhancing user trust and application security.

 

 

Monitoring and Health Checks

 

Utilizes Amazon CloudWatch for detailed metrics and logging, coupled with automatic health checks and recovery for maintaining optimal application performance.

 

 

Flexible Configuration

 

Allows the use of environment variables for easy application configuration without modifying code, accommodating dynamic changes like database connections or API keys.

 

 

Simple Pricing

 

Offers a straightforward pricing model based on compute and memory usage, aligning costs closely with resource consumption.

 

 

App Runner Use Cases

 

Web Hosting

 

Simplifies deploying web applications, APIs, microservices, and websites, integrating easily with services like Amazon RDS, ElastiCache, SQS, and SNS for enhanced functionality without infrastructure management.

 

 

Serverless Development

 

Ideal for serverless applications, App Runner automatically adjusts to traffic fluctuations, eliminating server management and enabling efficient resource utilization.

 

 

CI/CD

 

Facilitates continuous integration and deployment by connecting with SCM tools such as GitHub and Bitbucket, supporting automatic updates and streamlined code delivery.

 

 

Container Deployment

 

Supports deploying both container images and source code, aiding in containerizing existing applications or developing new ones for the AWS ecosystem.

 

 

Microservices and Prototyping

 

Serves microservices architectures by allowing independent deployment and scaling. It also accelerates prototyping, enabling rapid deployment and iteration.

 

 

E-commerce and CMS

 

Meets the demands of e-commerce platforms and content management systems, providing a scalable backend that adjusts to traffic changes and content management needs.

 

 

Educational Platforms and MVPs

 

Supports educational tools with variable usage patterns and assists startups in launching MVPs quickly, facilitating product validation with minimal upfront investment.

 

 

Analytics and Data Processing

 

Hosts applications for data processing and analytics dashboards, efficiently managing compute resources for background tasks and data visualization.

 

 

AWS Elastic Beanstalk

 

AWS Elastic Beanstalk, similar to AppRunner, allows the deployment of applications on AWS without managing the underlying infrastructure. However, Elastic Beanstalk utilizes EC2 infrastructure, to which you have access and can configure as needed. It supports a plethora of programming languages out of the box, provided in various forms, not only in containerized form as with AppRunner.

 

Elastic Beanstalk Key Features

 

Automated Resource Provisioning

 

Elastic Beanstalk automatically provisions resources like EC2 instances, load balancers, and auto-scaling policies, allowing for hassle-free application deployment without manual infrastructure management.

 

 

Comprehensive Language and Platform Support

 

Supports a broad array of languages and platforms, including Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker, catering to diverse development needs.

 

 

Simplified Deployment Process

 

Enables straightforward application deployment by allowing developers to upload code or container images, with Elastic Beanstalk handling the subsequent rollout to the provisioned environment.

 

 

Dynamic Scaling

 

Automatically adjusts resources to match traffic demands, scaling applications vertically (modifying EC2 instance sizes) or horizontally (adding more instances) as needed.

 

 

Robust Monitoring and Health Management

 

Features built-in application monitoring and health checks, ensuring high availability by restarting failed instances or deploying replacements.

 

 

Customizable Environments

 

Offers environment customization through .ebextensions files for detailed configuration adjustments, including security groups and EC2 instance types.

 

 

Continuous Deployment Integration

 

Facilitates continuous deployment from source control systems like GitHub, enabling streamlined code updates and deployments.

 

 

Environment Management

 

Allows management of resources as a single logical entity or environment, with support for creating multiple environments for development, testing, production, etc.

 

 

Lifecycle Resource Management

 

Manages the lifecycle of environment resources, including provisioning and cleanup during scaling events or environment creation and termination.

 

 

Elastic Beanstalk Use Cases

 

Web Application Hosting

 

Elastic Beanstalk facilitates the effortless deployment and management of web applications, eliminating the need for manual infrastructure provisioning and management, including servers and load balancers.

 

 

Continuous Application Deployment

 

Offers seamless integration with source control systems such as GitHub, enabling continuous deployment and updates to applications with minimal downtime.

 

 

Scalability

 

Features automatic scaling capabilities, adjusting the number of instances according to incoming traffic to maintain optimal application performance and availability.

 

 

Application Migration

 

Simplifies the migration process for existing applications from on-premises or other hosting solutions to AWS, ensuring a smooth transition to the cloud environment.

 

 

AWS Services Integration

 

Allows applications to easily integrate with a wide range of AWS services, including RDS for databases, S3 for storage, and SQS for messaging and queue management, enhancing application functionality.

 

 

Simplified Management

 

Manages all underlying infrastructure tasks such as provisioning, patching, and monitoring, freeing developers to focus on coding and innovation withou the burden of server management.

 

 

Amazon Elastic Container Service (ECS)

 

The key features of both ECS and EKS were already discussed in the previous article "The orchestration of containers on AWS"; therefore, only the use cases will be covered here.

 

ECS Use Cases

 

Microservices Deployment

 

Facilitates the scalable orchestration of microservices-based applications using containers, allowing for independent deployment and inter-service communication.

 

 

CI/CD Integration

 

Seamlessly integrates with version control systems for continuous deployment and automated updates, enhancing the software development lifecycle.

 

 

Serverless Architectures

 

With the Fargate launch type, ECS offers a serverless option, managing server provisioning and scaling so users can focus on container management.

 

 

Batch and Data Processing

 

Ideally suited for executing batch jobs and data processing tasks, ECS schedules and auto-scales tasks without the need for dedicated servers, efficiently handling large data volumes.

 

 

Backend Services for Applications

 

Provides a reliable, secure, and scalable hosting solution for backend services and microservices supporting front-end interfaces, ensuring high availability.

 

 

Media Processing

 

Utilized for high-throughput processing of unstructured data, including video, image, and audio files for transcoding and metadata extraction, distributing tasks for enhanced performance.

 

 

Amazon Elastic Kubernetes Service (EKS)

 

Please refer to the previous paragraph for information on the key features of EKS.

 

EKS Use Cases

 

Microservices Deployment

 

Streamlines deploying containerized microservices, offering independent deployment and inter-service communication, enhanced by Kubernetes' service discovery and load balancing capabilities.

 

 

CI/CD Integration

 

Seamlessly integrates with version control systems such as GitHub for automated deployments, supporting continuous integration and delivery workflows for Kubernetes applications.

 

 

Serverless Computing with Fargate

 

Offers a serverless option by integrating with AWS Fargate, removing the need for server provisioning and management, allowing developers to concentrate on their containerized applications.

 

 

Machine Learning Workloads

 

Supports machine learning frameworks like TensorFlow, MXNet, and PyTorch, with GPU support for handling complex tasks efficiently, facilitating scalable ML pipelines and inference processes.

 

 

High Availability Applications

 

Ensures high application availability across multiple availability zones, utilizing Kubernetes' self-healing features and AWS services for robust load balancing and resilience.

 

 

Resilient Microservices Architectures

 

Enhances microservices architectures with Kubernetes' native features and AWS services like Cloud Map or VPC Lattice for resilient service discovery and networking.

 

 

Efficient Batch Processing

 

Manages batch jobs effectively, dynamically allocating resources for data processing, scientific simulations, and other batch-oriented workloads.

 

 

Hybrid and Multi-Cloud Flexibility

 

Supports hybrid and multi-cloud deployments, allowing Kubernetes workloads to run on AWS, on-premises, or across other cloud platforms, ensuring flexibility and avoiding vendor lock-in.

 

 

Disaster Recovery

 

Leverages Kubernetes and AWS services integration for disaster recovery strategies, maintaining high availability and data integrity across applications.

 

 

IoT and Edge Computing

 

Suitable for IoT and edge computing, enabling Kubernetes workloads to be deployed closer to data sources or users, reducing latency and optimizing data processing.

 

 

Big Data and Analytics

 

Facilitates running big data and analytics workloads, using Kubernetes to orchestrate scalable data processing applications for analytics and real-time data insights.

 

 

Gaming Platforms

 

Delivers the scalability and performance required for online gaming applications, ensuring smooth and responsive experiences for a large number of simultaneous users.

 

 

Amazon EMR

 

Amazon EMR provides a platform where Big Data frameworks can be deployed on fully managed underlying hardware.

 

EMR Key Features

 

Fully Managed Big Data Frameworks

 

EMR automates the deployment and management of popular big data frameworks such as Hadoop, Spark, Presto, and Hive, along with their dependencies, eliminating the need for manual setup and management.

 

 

Quick and Easy Cluster Setup

 

Launching EMR clusters is straightforward and can be done within minutes through a few clicks in the AWS Management Console or via API calls, provisioning all necessary infrastructure swiftly.

 

 

Dynamic Auto-Scaling

 

To optimize costs and performance, EMR supports the automatic scaling of clusters, adjusting their size based on the workload by adding or removing instances as required.

 

 

Seamless AWS Integration

 

EMR clusters can seamlessly access and process data stored in AWS services like S3, DynamoDB, and Redshift, facilitating efficient data analysis workflows.

 

 

Robust Security Measures

 

Security in EMR is multi-faceted, including EC2-based cluster security, data encryption at rest via AWS KMS, and network isolation with Amazon VPC.

 

 

Comprehensive Monitoring and Logging

 

Integrated with CloudWatch, EMR provides detailed monitoring, logging, and debugging tools for both applications and infrastructure.

 

 

Flexible Computing Options

 

EMR offers various computing options to meet diverse workload needs, including On-Demand and Spot Instances across different EC2 instance types.

 

 

Performance Optimization

 

With continuous updates and enhancements to the underlying big data frameworks, EMR ensures optimized performance and efficiency of data processing tasks.

 

 

Broad AWS Ecosystem Integration

 

The service is designed to work cohesively with AWS's data lakes, analytics, and machine learning services, enabling the development of extensive big data applications.

 

 

Highly Customizable Environment

 

Users have the flexibility to tailor cluster settings, select specific software versions, choose EC2 instance types, and run custom applications in conjunction with standard big data frameworks.

 

 

EMR Use Cases

 

Efficient Large-Scale Data Processing

 

EMR excels in processing enormous datasets using distributed computing with optimized versions of Hadoop and Spark, ideal for log analysis, clickstream analytics, and machine learning on extensive datasets.

 

 

Data Warehousing and Lakes

 

It facilitates the construction of data warehouses and analytics data lakes, enabling data ingestion from diverse sources into efficient formats like Parquet or ORC, and supports querying with Hive or Spark SQL over scalable S3 storage.

 

 

Batch and Stream Processing

 

Suited for regular batch jobs to transform data, and real-time stream processing from sources like social media or IoT devices, EMR enhances timely data analysis and decision-making.

 

 

Interactive Queries and Data Exploration

 

Offers the capability for analysts to perform interactive SQL queries using Spark SQL or Presto, and data scientists to explore data, build, and deploy machine learning models using notebook environments like Jupyter.

 

 

Comprehensive Analytics and ML

 

Supports complex data analytics, predictive modeling, and AI by running analytical queries and developing machine learning models with frameworks such as Apache Spark MLlib and TensorFlow.

 

 

Diverse Application Support

 

From log aggregation and monitoring, financial risk modeling, and fraud detection, to powering recommendation engines and processing geospatial data for location-based services, EMR serves a broad spectrum of use cases.

 

 

Specialized Workloads

 

It's also applied in genomics for processing genetic data, contributing to research in life sciences, and in interactive data visualization and exploration for insights.

 

 

References

 

Compute services

What is Compute? - Enterprise Cloud Computing Explained - AWS

Compute on AWS - Amazon Web Services

Free Compute Services on AWS

What is The Monte Carlo Simulation? - AWS

Spot Instances

Spot instances for HPC workload: use cases

When to Use Spot Instances - Overview of Amazon EC2 Spot Instances

Amazon EC2 Dedicated Hosts FAQs - Amazon Web Services | What are the benefits of using Dedicated Hosts?

Placement groups - Amazon Elastic Compute Cloud

What is Amazon Lightsail? - Amazon Lightsail

What are the differences between Amazon EC2 and Amazon Lightsail?

What is AWS Lambda? - AWS Lambda

Common Lambda application types and use cases - AWS Lambda

Amazon ECS on AWS Fargate

AWS Fargate - Amazon EKS

Welcome - AWS Outposts

AWS Batch Documentation

AWS Batch Use Cases

What is AWS App Runner?

Getting started with App Runner

AWS Elastic Beanstalk Documentation

Using Elastic Beanstalk with other AWS services - AWS Elastic Beanstalk

AWS Elastic Beanstalk FAQs - Amazon Web Services (AWS) | Who should use AWS Elastic Beanstalk?

Amazon EMR Documentation