Welcome to Day 46 of the #100DaysOfDevOps Challenge! Today, we will delve into AWS EC2 and explore the power of launch templates, instance types, and AMIs for streamlined deployment and management of EC2 instances. and Auto Scaling Groups.

It is very difficult to predict how much computing power one might require for an application that you might have just launched. There can be two scenarios, you may over-estimate the requirement, and buy stacks of servers that will not be of any use, or you may under-estimate the usage, which will lead to the crashing of your application. In this AWS EC2 Tutorial, we will understand all the key concepts using examples and also perform hands-on to launch an Ubuntu instance.

What is AWS EC2?

Amazon Elastic Compute Cloud, EC2 is a web service from Amazon that provides re-sizable compute services in the cloud.

How are they re-sizable?

They are re-sizable because you can quickly scale up or scale down the number of server instances you are using if your computing requirements change.

That brings us to our next question.

What is an Instance?

An instance is a virtual server for running applications on Amazon’s EC2. It can also be understood like a tiny part of a larger computer, a tiny part which has its own Hard drive, network connection, OS etc. But it is actually all virtual. You can have multiple “tiny” computers on a single physical machine, and all these tiny machines are called Instances.

Difference between a service and an Instance?

Let’s understand it this way:

• EC2 is a service along with other Amazon Web Services like S3 etc.

• When we use EC2 or any other service, we use it through an instance, e.g. t2.micro instance, in EC2 etc.

Why AWS EC2?

Why not buy your stack of servers and work independently? Because, suppose you are a developer, and since you want to work independently you buy some servers, you estimate the correct capacity, and the computing power is enough. Now, you have to look after the updation of security patches every day, you have to troubleshoot any problem that might occur at a back-end level in the servers ad so on. These are all extra chores that you will be doing or maybe you will hire someone else to do these things for you.

But if you buy an EC2 instance, you don’t have to worry about any of these things as it will all be managed by Amazon; you just have to focus on your application. That too, at a fraction of the cost that you were incurring earlier! Isn’t that interesting?

Let’s understand Cost Savings using an example.

Suppose instead of taking AWS EC2, we consider taking a dedicated set of servers, so, what all we might have to face:

• Now for using these servers, we have to hire an IT team which can handle them.

• Also, having a fault in the system is unavoidable, therefore we have to bear the cost of getting it fixed, and if you don’t want to compromise on your up-time you have to keep your systems redundant to other servers, which might become more expensive.

• Your own purchased assets will depreciate over the period of time, however, as a matter of fact, the cost of an instance have dropped more than 50% over a 3-year period, while improving processor type and speed. So eventually, moving to Cloud is all more suggested.

• For scaling up we have to add more servers, and if your application is new and you experience sudden traffic, scaling up that quickly might become a problem.

These are just a few problems and there are many other scenarios which make the case for EC2 stronger!

Let’s understand the types of EC2 Computing Instances:

Computing is a very broad term, the nature of your task decides what kind of computing you need.

Therefore, AWS EC2 offers 5 types of instances which are as follows:

1. General Instances

For applications that require a balance of performance and cost.

E.g email responding systems, where you need a prompt response as well as it should be cost-effective since it doesn’t require much processing.

2. Compute Instances

For applications that require a lot of processing from the CPU.

E.g analysis of data from a stream of data, like a Twitter stream

3. Memory Instances

For applications that are heavy in nature, therefore, require a lot of RAM.

E.g when your system needs a lot of applications running in the background i.e multitasking.

4. Storage Instances

For applications that are huge in size or have a data set that occupies a lot of space.

E.g When your application is of huge size.

5. GPU Instances

For applications that require some heavy graphics rendering.

E.g 3D modeling etc.

Now, every instance type has a set of instances which are optimized for different workloads:

1. General Instances

• t2

• m4

• m3

2. Compute Instances

• c4

• c3

3. Memory Instances

• r3

• x1

4. Storage Instances

• i2

• d2

5. GPU Instances

• g2

Now let’s understand the kind of work that each instance is optimized for, in this AWS EC2 Tutorial.

EC2

EC2 stands for Elastic Compute Cloud. It is a web service provided by Amazon Web Services (AWS).

EC2 provides scalable computing capacity, allowing users to quickly provision and configure virtual machines called instances.

With EC2, users can choose the type of instance they need, such as the amount of CPU, memory, storage, and networking capacity. Instances can be launched from pre-configured Amazon Machine Images (AMIs) that contain various operating systems and software configurations, or users can create their own custom AMIs.

EC2 provides a flexible and scalable infrastructure for running applications in the cloud, allowing users to pay only for the resources they consume and easily scale their computing capacity up or down as needed.

The below image depicts the lifecycle of an Instance:

Instances can be launched using the launch instance wizard or using a launch template.

Launch Templates in EC2

A launch template is a feature provided by Amazon EC2 that allows you to create reusable configurations for launching instances.

You can create a launch template that contains the configuration information to launch an instance. You can use launch templates to store launch parameters so that you do not have to specify them every time you launch an instance.

Launch templates support versioning, which means you can create multiple versions of a template to track changes over time. If you do not specify a version, the default version is used. You can set any version of the launch template as the default version — by default, it’s the first version of the launch template.

Instance Types

When you launch an instance, the instance type that you specify determines the hardware of the host computer used for your instance.

Instance types are named based on their family, generation, additional capabilities, and size.

• The first position of the instance type name indicates the instance family, for example c.

• The second position indicates the instance generation, for example 5.

• The remaining letters before the period indicate additional capabilities, such as instance store volumes.

• After the period (.) is the instance size, which is either a number followed by size, such as 9xlarge, or metal for bare metal instances.

The different types of Instances provided by AWS are:

General Purpose

These instances provide a balance of CPU, memory, and network resources. They are suitable for a wide range of workloads, including web servers, small databases, development environments, and other applications that require a balance of compute power and memory.

Examples: t3, m5

Compute Optimised

These instances are designed for compute-intensive workloads that require high CPU performance. They offer a high ratio of vCPUs to memory and are ideal for applications like scientific modeling, batch processing, gaming servers, and highly scalable front-end fleets.

Examples: c5

Memory Optimised

Memory-optimized instances are designed for memory-intensive workloads such as in-memory databases, real-time analytics, and high-performance computing. They offer a large amount of RAM relative to CPU resources, allowing efficient processing of data-intensive applications.

Examples: r5

Storage Optimised

Storage-optimized instances are designed for workloads that require high storage capacity and high sequential read/write performance. They are suitable for big data processing, data warehousing, and other applications that require fast and large-scale data access.

Examples: i3, d2.

GPU Instances

GPU instances are equipped with powerful graphics processing units (GPUs) that accelerate applications requiring parallel processing and high-performance computing. They are used for machine learning, deep learning, video rendering, and other GPU-intensive workloads.

Examples: p3, g4.

FPGA Instances

FPGA (Field Programmable Gate Array) instances are specialized instances that provide programmable hardware acceleration. They are suitable for applications that require custom hardware acceleration and can be programmed to perform specific tasks efficiently.

Examples: f1

Instance Sizes

AMI

AMI stands for Amazon Machine Image. It is a pre-configured template that contains the necessary information to launch an instance in Amazon EC2. An AMI serves as the building block for creating virtual servers in the cloud.

AMIs are available as Amazon-provided AMIs, AWS Marketplace AMIs, and Custom AMIs.

An AMI includes Root Volume, Instance Configuration, Permissions, Block Device Mappings, and Launch Permissions.

Task 1 - Create Linux instance and Launch Instances

Step 1: Sign in to the AWS Management Console

To create an EC2 instance, you first need to sign in to the AWS Management Console. If you don't already have an AWS account, you'll need to create one. Once you're signed in, navigate to the EC2 dashboard and Launch an instance.

Step 2: Choose a name of your instance

Select a name of your instance as per your likability

Step 3: Choose an Amazon Machine Image (AMI)

Selecting AMI for your instance

An Amazon Machine Image (AMI) is a pre-configured virtual machine that serves as a template for your EC2 instance. You'll be prompted to choose an AMI from a list of available options. You can choose from Amazon Linux, Ubuntu, Windows, and many other options.

Step 4: Choose an Instance Type

An instance type determines the computing resources (CPU, RAM, storage, etc.) available to your EC2 instance. There are a variety of instance types to choose from, ranging from small and low-cost to large and high-performance. Select the instance type that best fits your needs and budget.

Step 5: Create a key pair

Create a key pair if you have never created one and store it in a safe place because it will act as a key to log in to your instance.

Step 6: Configure Security Group

Security groups act as virtual firewalls for your EC2 instance, controlling inbound and outbound traffic. You can configure security groups to allow or deny traffic from specific IP addresses, protocols, and ports. In this step, you'll need to create a new security group or select an existing one.

Step 7: Add Storage

EC2 instances require storage for the operating system, applications, and data. In this step, you can add and configure storage volumes for your instance. You can choose from different types of storage, including Amazon Elastic Block Store (EBS) volumes and instance store volumes.

Step 8: Review and Launch

Before launching your instance, review all the details to make sure everything is correct. You can also modify any settings that need to be changed. Once you're ready, click the "Launch" button to start your EC2 instance.

Step 9: Connect to Your Instance

After launching your instance, you can connect to it using various methods, such as SSH or Remote Desktop Protocol (RDP). You can also use the AWS Systems Manager Session Manager to connect to your instance securely without the need for a public IP address.

Linux/Ubuntu servers we can connect using Bash shell or Putty.

1. First we should secure.PEM file using the “chmod 400” command

2. We can copy commands from the connection tab for the Server connection.

3. admin/username: Ubuntu

AWS Linux Server connection using Putty.

Here are the general steps for connecting using PuTTY:

1. Open PuTTY.

2. Enter the hostname or IP address of your Linux server in the "Host Name" field.

3. Make sure the port is set to 22 (default for SSH).

4. Choose the connection type as "SSH."

5. On the left side, navigate to "Connection" -> "Data" and enter the username in the "Auto-login username" field.

6. Optionally, you can save the session settings for future use by giving it a name under "Session" and clicking "Save."

7. Click "Open" to initiate the connection.

If you are using key pair authentication, you might need to configure the private key file under "Connection" -> "SSH" -> "Auth" by browsing to the location of your private key.

Once you click "Open," PuTTY will prompt you for your password or use the key pair for authentication, depending on your configuration.

When connecting to a Linux server using PuTTY, the login prompt is asking for the username. You need to enter the username for the account on the Linux server that you want to log in with.

By default, if you are using an Amazon EC2 instance, the username for connecting to a Linux instance varies depending on the Linux distribution:

• For Amazon Linux: ec2-user

• For Ubuntu: ubuntu

• For CentOS: centos

• For Debian: admin or root

Creating an EC2 instance in AWS is a simple and straightforward process. With just a few clicks, you can launch a virtual machine in the cloud and start using it right away. By following the steps outlined in this guide, you can create your own EC2 instance in no time.

Automation in EC2

Automation is a key aspect of utilizing the full potential of Amazon EC2 (Elastic Compute Cloud). With EC2, you gain access to a secure, reliable, high-performance, and cost-effective computing infrastructure that can cater to even the most demanding business needs. However, to truly harness the power of EC2, automation becomes essential. By automating various tasks and processes, you can enhance operational efficiency, reduce manual errors, and ensure consistent configurations across your EC2 instances.

Automation enables you to streamline workflows such as instance provisioning, configuration management, scaling, and monitoring. Instead of performing these tasks manually, automation tools and practices allow you to define and execute them programmatically. This not only saves time but also ensures consistency and reproducibility in your infrastructure. Through automation, you can create scripts, templates, or use specialized tools to define and automate the deployment and management of EC2 instances, making it easier to handle large-scale deployments, achieve rapid scaling, and adapt to changing workload demands.

Launch template in AWS EC2

A launch template is a powerful feature in AWS EC2 that simplifies the process of launching instances with consistent configurations. It allows you to define a set of pre-configured parameters and settings required to launch instances, such as the Amazon Machine Image (AMI) ID, instance type, network settings, storage options, and security groups. By creating a launch template, you can capture all these configurations in a reusable template, eliminating the need to manually specify them each time you launch an instance.

Launch templates offer flexibility and convenience. They enable you to create standardized configurations for different use cases, ensuring consistency and reducing the chances of configuration errors. Moreover, launch templates can be shared across teams or organizations, promoting collaboration and standardization in infrastructure deployments. With launch templates, you can easily replicate and launch instances with consistent settings, making it simpler to manage and scale your EC2 infrastructure.

Task 2 - Create a Launch Template and Deploy Instances

1. Go to the Amazon EC2 console in the AWS Management Console.

2. Navigate to "Launch Templates" in the left navigation pane and click on "Create launch template".

3. Provide a name and description for your launch template.

4. Under "Choose an Amazon Machine Image (AMI)", select "Amazon Linux 2023" as the AMI.

5. Scroll down to the "Instance type" section and choose "t2.micro" as the instance type.

6. In the "User data" section, enter the user data script that installs Jenkins and Docker (you can use the script from Day 39).

7. Configure other settings such as network, storage, security groups, and tags according to your requirements.

8. Click on "Create launch template".

   

To create 3 instances using the launch template:

1. Go back to the Amazon EC2 console.

2. Navigate to "Instances" in the left navigation pane and click on "Launch instances" dropdown menu. Select "Launch instance from template"

   

3. Choose the launch template you created earlier from the list.

4. In the "Number of instances" field, specify "3" as the number of instances to be launched.

5. Review and configure other settings as needed.

6. Click on "Launch instances" to start the instances.

   

   

Task 2 - Create an auto-scaling group:

1. Go back to the Amazon EC2 console.

2. Navigate to "Auto Scaling Groups" in the left navigation pane and click on "Create Auto Scaling group".

3. Configure the basic settings for the auto-scaling group, such as the launch template, desired capacity, and minimum and maximum instances.

4. Set up scaling policies based on your requirements, such as scaling based on CPU utilization or other metrics.

5. Configure other advanced options such as health checks, notifications, and tags.

6. Review and confirm the settings.

7. Click on "Create Auto Scaling group" to create the auto-scaling group.

   

Creating an auto-scaling group allows your instances to automatically scale based on the defined policies, ensuring that your infrastructure can handle varying workloads efficiently.

Now, let's run a stress test to ensure the Auto Scaling Group is working as expected:

1. Download and install the stress test tool:

     sudo amazon-linux-extras install epel -y
     sudo yum install stress -y
   

2. Launch the stress test in background (for 800 seconds in that example)

     sudo stress --cpu 8 --timeout 800 &
   

3. Check the stress test processes with the top command

     top
   

   

4. Once the instance CPU Utilization hits a value above the 50% threshold we've set up in the Auto Scaling Group, it will create a new instance. The maximum numbers of instances we can have is set up by the "Maximum Capacity" value in the ASG.

   

   

   

5. We've just proved the ASG is working as expected and it will increase the capacity based on the CPU utilization.

Congratulations on completing Day 40 of the #90DaysOfDevOps Challenge. Today, we explored automation in AWS EC2, learned about launch templates, different instance types, and AMIs, empowering us to optimize our infrastructure and efficiently deploy applications. Stay tuned for Day 41, where we will learn how to set up an Application Load Balancer with AWS EC2.