MASTER-SLAVE LOAD TESTING USING AWS EC2 SPOT INSTANCES

Load testing using JMeter from a single server generally yields the inaccurate result which doesn’t meet the real-time scenario. When performing load test from a single server, there are some network bottlenecks which restrict JMeter to perform an accurate load test. If the number of users exceeds a limit, the network starts getting choked up leading to an incorrect result. It is always better to perform load test from different sites by sending concurrent users to your application which not only brings the level of accuracy to your load test but also give flexibility to check the performance of your web application in real time scenario.

Use case

When we perform load test using EC2 servers, we need to pay the full hour cost of the servers in spite of knowing that our load test will take less than an hour. When it comes to the master-slave setup of load testing, you might be needing 5-10 slave servers to perform an efficient load test. This problem can be solved by using the spot instances which can be available to you in 60-70% lesser cost than a previous one. Spot instances in this scenario will fit best in the picture because it doesn’t matter if one of your slave instances got terminated due to high pricing, you can start your load test again with an ease. In this blog, I will be demonstrating how we can perform distributed load testing using master-slave setup in the same subnet of a VPC.

Master-Slave Jmeter Load Testing

In Master-Slave JMeter load testing, one instance behaves like a master (generally a small-sized  instance) and multiple high-performance servers behave like slave servers. Slave servers actually perform the load test by sending requests on to the web application. The master server triggers the load test job and gives aggregated output in XML format which can further be converted into HTML format using Apache Ant tool.

Below is a bash script which will set up the master-slave configuration using spot- instances in a single public subnet.

Prerequisites:-
1. Load testing  JMX file
2. Apache Ant configured Master Server.
3. AWS-CLI
4. A build.xml for Apache Ant, required to present in the target directory of JMeter to view the result in a web browser after converting the result from XML to HTML.

Script Overview:-

The script will perform following tasks step by step after being run from master JMeter server:-

1. Choosing the number and type of slave servers.
2. Bidding of slave servers by showing you the last hour pricing of the instances in all the Availability zones.
3. Configuring JMeter on slave servers and preparing them for ready to use.
4. Performing load test in a master-slave configuration.
5. Giving XML output of the load test.

Below is a script named “user_data_for_loadtesting.sh” which will be passed as a user data to the slave servers when the spot requests will be accepted. The script will install JMeter in the slave servers and make them ready to use by running the script JMeter server.

[js]
#!/bin/bash
sudo apt-get update
sudo apt-get install jmeter -y
sudo sed -i ‘s/remote_hosts=127/#remote_hosts=127/’ /usr/share/jmeter/bin/jmeter.properties
sudo /usr/share/jmeter/bin/jmeter-server
[/js]

Below is the main script named “perform_load_test.sh” to perform the whole task.

[js]
#!/bin/bash
current_date=`date +%Y-%m-%dT%H:%M:%S`
hour_ago_date=$(date +%Y-%m-%dT%H:%M:%S -d "1hour ago")
echo "= + = + = + = +WELCOME TO AUTOMATED LOAD TEST USING SPOT INSTANCES= + = + = +"
echo "choose instance type:"
echo "1.m3.medium"
echo "2.m3.large"
echo "enter choice:"
read ch
case $ch in
1) instance_type=m3.medium;;
2) instance_type=m3.large;;
*) invalid option;;
esac

echo "SPOT PRICE HISTORY FOR $instance_type IN LAST HOUR IS :- "
aws ec2 describe-spot-price-history –region us-east-1 –instance-types m3.medium –filters Name="product-description",Values="Linux/UNIX" –start-time $hour_ago_date –end-time $current_date –query ‘SpotPriceHistory[].[AvailabilityZone,SpotPrice]’ –output text | tail -n 4

echo "enter your spot price::"
read price
#Here is a need to encode the userdata script in base64 format to pass it as a user data.

userdata_encoded_slave="$(cat ~/user_data_for_loadtesting.sh | base64 -w 0)"

echo "enter no. of slave instances required:: "
read number
req_id=($(aws ec2 request-spot-instances –region us-east-1 –spot-price $price –launch-specification "{ \"KeyName\": \"kew\", \"ImageId\": \"ami-d05e75b8\" , \"InstanceType\": \"m3.medium\" , \"UserData\": \"$userdata_encoded_slave\" , \"SecurityGroupIds\": [\"sg-8b451aef\"] }" –query ‘SpotInstanceRequests[].SpotInstanceRequestId’ –output text –instance-count $number))

for r in "${req_id[@]}"
do
aws ec2 describe-spot-instance-requests –spot-instance-request-ids $r –query ‘SpotInstanceRequests[].[SpotInstanceRequestId,State]’ –output text | grep active
status=$?
while [ $status -ne 0 ]
do
echo "Checking for approval..Please wait"
sleep 8
aws ec2 describe-spot-instance-requests –spot-instance-request-ids $req_id –query ‘SpotInstanceRequests[].[SpotInstanceRequestId,State]’ –output text | grep active
status=$?
done

echo "************! ! ! ! ! ! CONGRATULATIONS !!!!….YOUR REQUEST HAS BEEN ACCEPTED !!!!!!!!*************************"

inst_id=`aws ec2 describe-spot-instance-requests –spot-instance-request-ids $r –query ‘SpotInstanceRequests[].InstanceId’ –output text`

ip=`aws ec2 describe-instances –instance-ids $inst_id –query ‘Reservations[].Instances[].NetworkInterfaces[].Association[].PublicIp[]’ –output text`

echo "The IP of allocated instance is :- $ip"
r_ips=$r_ips,$ip #formatting for jmeter command to pass remote IPs
done
remote_ips=`echo $r_ips | sed -e ‘s/,//’`
sudo sed -i ‘s/remote_hosts=127/#remote_hosts=127/’ /usr/share/jmeter/bin/jmeter.properties
echo "Performing load testing ……..Please wait"
echo "Initializing Remote host..Please wait.."
sleep 210
#to perform load test in master slave, the command is
#jmeter -n -t <jmx file> -l <result.xml> -R xxx.xxx.xxx.xxx, xxx.xxx.xxx.xxx (remote hosts)
jmeter -n -t /home/ubuntu/<jmx-file.jmx> -l /home/ubuntu/result.xml -R $remote_ips
[/js]

In the end, you will get an XML file named result.xml which can easily be converted in HTML format using Ant Apache tool or some other tool.