Configuring and testing AWS Elastic Load Balancer

Load balancing is an essential component for the scalability and fault tolerance of web applications. Major cloud computing providers have different offerings for load balancing.
In this post I'll explore AWS's (Amazon Web Services) ELB (Elastic Load Balancing) feature, and test it to see how it distributes the load on front-end web servers, and in case of unavailability of one of the front-end servers, how traffic is directed to the healthy instance(s).

I'll use Linux based image, but the concepts apply to Windows images. I assume that the reader has the basic knowledge on how to create an AWS account and create EC2 (Elastic Compute Cloud) virtual machine. If not, don't worry, following the steps below will give you a good understanding.

So the experiment goes as follows:

1- Create a base image for front-end web servers: 

 

1. Go to AWS console and select "Launch Instance", from the list of images, select "Ubuntu Server 14.04 LTS".
2. Complete the wizard till you reach the "Configure Security Group" step. In this is the step we select the proper ports we need AWS to open. Select SSH (22) to connect to the instance to configure it, and HTTP (80) to serve web traffic.
3. When you're prompted to select the key pair, make sure to choose an existing one you have already downloaded or create a new one and keep it in a safe place.
4. Then Launch the instance.

Note: When I first stared using AWS, and being from a windows background, the term "Security Group" was a bit confusing to me, it's about firewall rules not security groups in the sense of Active Directory Groups.

2- Configure Apache web server

The image does not have a web server installed by default, so I'll SSH into the instance and install it.

If you're using MAC or Linux, you should be able to run SSH directly. For Windows users, you can use Putty.

1. Copy the public IP of the running instance you just created.
2. Use SSH to connect using this command:  ssh -l ubuntu -i . for example: ssh 54.72.151.182 -l ubuntu -i mykey.pem . note that ubuntu is the username for the image we created this machine from. the .pem file acts as a password.
3. Now we are inside the instance. It's time to install and configure Apache:

sudo su
apt-get install apache2
sudo a2enmod cgi
service apache2 restart

The above commands simply do the following:

• Elevate privileges to run as a super user to be able to install software.
• Install apache using the package manager.
• Enable CGI, I'll show you why later
• Restart apache so that CGI configuration takes effect.

Now it's time to test the web server. Visit http://INSTANCE_IP and you should be welcomed with the default apache home page.

3- Create a script to identify the running instance

To test ELB, I need to identify which instance served the request just by looking into the response to a web request. Now I have 2 options: Create static pages on each web fron-end or create some dynamic content that identifies the instance. And I prefer the latter option as I'll use the same image for all front-ends.

EC2 has a nice feature called instance metadata. It's an endpoint accessible from within EC2 instances that can be called to get information about it. From SSH terminal try:

curl http://169.254.169.254/latest/meta-data/

A list of available meta-data will be shown:

ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
hostname
instance-action
instance-id
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-hostname
public-ipv4
public-keys/
reservation-id
security-groups
services

Appending any of them to the URL will show the value. For example:


curl http://169.254.169.254/latest/meta-data/public-hostname
curl http://169.254.169.254/latest/meta-data/public-ipv4

And I'll use these two meta-data items to identify the instances by showing them within a bash script and then serve it from apache. cd into /usr/lib/cgi-bin

cd /usr/lib/cgi-bin

This is the default location that apache uses to serve CGI content. That's why I enabled CGI in a previous step.
in that folder I'll create a bash script that shows the output of the meta-data. use any text editor. For example run nano in the command line and paste the below script:

#!/bin/bash

echo "Content-type: text/text"
echo ''
echo 'Host name:'
curl http://169.254.169.254/latest/meta-data/public-hostname
echo ''
echo 'Public IP:'
curl http://169.254.169.254/latest/meta-data/public-ipv4

If using nano, ctrl+X, y. save as metadata.sh

Now we need to grant execute permission on this file:

chmod 755 /usr/lib/cgi-bin/metadata.sh

To test the configuration, browse to http://INSTANCE_IP/cgi-bin/metadata.sh
My results look like:

Host name:
ec2-54-72-151-182.eu-west-1.compute.amazonaws.com
Public IP:
54.72.151.182

Note: I'm not advising using bash scripts in production web sites. It just was the easiest way to spit out info returned from the meta-data endpoints with minimal effort.

4- Create 2 more front-ends

Now we have an identifiable instance. Let's create more of it.

1. Stop the instance from the management console
2. After the instance has stopped, right click -> image -> create image.
3. Choose and appropriate name and save.
4. Navigate to AMI (Amazon Machine Image) and check the creation status of the image.
5. Once the status is available click launch
6. In the launch instance wizard, select to launch 2 instances
7. Select the same security group as the one used before, it will have both 22 and 80 ports open.
8. Start the original instance. 
9. Now we have 3 identical servers.
10. Using the IP address of any instance, navigate to the CGI script, for example: http://52.17.134.221/cgi-bin/metadata.sh

Note that most probably the IP of the first instance is now different after restart.

5- Create an ELB instance

1. In AWS console, navigate to "Load Balancers".
2. Click "Create Load Balancer"
3. Make sure it's working on port 80
4. Select the same security group
5. In the health check, in the ping path, enter "/". This means that ELB will use the default apache page for health check. In production, it might not be a good idea to make your home page the health check page.
6. For quick testing, make the "Healthy Threshold" equal to 3.

Now a bit of explanation is required. This configuration tells ELB to check for the healthiness of a front-end instance every 30 seconds. A check is considered successful if the server responds in 5 seconds.

If a healthy instance does not respond with that period for 2 consecutive failures, it's considered unhealthy. And similarly, an unhealthy instance is considered healthy again if it responds to the check 3 consecutive times.

Now select the 3 instances to use for load balancing. And wait until the ELB instance is created and the 3 instances in the "instances" tab are shown InService.

Now in the newly create ELB, select the value of the DNS name (like test-elb-1856689463.eu-west-1.elb.amazonaws.com) and navigate to the URL of the metadata page. My url looked like:
http://test-elb-1856689463.eu-west-1.elb.amazonaws.com/cgi-bin/metadata.sh

The data displayed in the page will belong to the instance that actually served the request. Refresh the page and and see how the response changes. In my case ELB worked in a round robin fashion and the responses where:

Host name:
ec2-52-17-134-221.eu-west-1.compute.amazonaws.com
Public IP:
52.17.134.221


Host name:
ec2-52-16-189-41.eu-west-1.compute.amazonaws.com
Public IP:
52.16.189.41


Host name:
ec2-52-17-65-93.eu-west-1.compute.amazonaws.com
Public IP:
52.17.65.93

Inspect the network response using F12 tools and note the headers:

HTTP/1.1 200 OK
Content-Type: text/text
Date: Sat, 16 May 2015 19:12:38 GMT
Server: Apache/2.4.7 (Ubuntu)
transfer-encoding: chunked
Connection: keep-alive

Note: nothing special as there is no session affinity.

6- Bring an instance down

Now, let's simulate an instance failure. Let's simply stop the apache service on one of the 3 front-ends. So ssh into one of the 3 instances and run:

sudo service apache2 stop

Refresh the page pointing to the ELB url, note that after a few seconds, you only get responses from the 2 running instances. After about 1 minute, the instance is declared OutOfService in the Instances tab of ELB.

 

7- Bring it back!

This time, turn on apache service by running:

sudo service apache2 start

Wait about one and half minutes, the instance is back to InService status and you start to get responses from it.

The "Healthy Hosts ( Count )" graph shows a very good representation of what happened:

8- Turn them all off!

They are costing you money, unless you are still under the free tier. It's recommended to terminate any EC2 and ELB instances that are no longer used.

Note:
If you intend to leave some instances alive, it's recommended to de-register the instance from ELB when shut down: http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-deregister-register-instances.html

 

Summary:

In this post, we've seen ELB in action using its basic settings. The round robin load balancing worked great and health check made our site available to users by eliminating unhealthy instances.
This works great with web applications that don't require session affinity, for applications that require it, well, that's another post.