Blog

How-To: Automating Your Testing with CloudTest and Jenkins

January 8, 2019 · by javier garza and salvador camacho reyes ·
Categories:

Akamai CloudTest gives you optimized testing capabilities for web and mobile apps, APIs, databases, and web services. Jenkins is one of the industry’s most popular open-source automation servers. In certain scenarios, you can combine the two of them for a superior automated testing solution.

First, let’s outline one of these scenarios.

Let’s say you have a CloudTest Test Composition* that includes a plan to run your testing at certain specified times and you want to:

  1. Bring up the CloudTest environment

  2. Bring up the necessary grid(s)

  3. Run your Test Composition

  4. Plot the results in Jenkins

  5. Bring down the necessary grid(s)

  6. And finally, bring down the CloudTest environment

*A CloudTest Test Composition is a multi-track interface where test clips can be placed at different times and tracks to create complex testing scenarios. This interface is similar to music and movie digital creation tools. Test Compositions are created by dragging and dropping Test Clips directly onto a timeline or in a sequence based on when they should execute.

The entire scenario above can be automated with CloudTest and Jenkins.

Note: To do the following CloudTest/Jenkins activity, you must have:

  • A CloudTest Manager account with access to the environment you want to launch
  • A CloudTest account of the environment where you want to bring up grids and run the test

First, we need to configure Jenkins for using CloudTest; we’ll start by installing some Jenkins plugins that will help.

Installing the CloudTest Jenkins plugin

In Jenkins, go to Jenkins -> Manage Jenkins -> Manage Plugins:

manage jenkins

Then click on the Available tab:

available

And search for CloudTest:

search

Then click the “SOASTA CloudTest” checkbox and click the “Install without restart” button if you are running jobs, or “Download now and install after restart” if you’re not running jobs:

download now

If you clicked the “Install without restart” button, then check the box “Restart Jenkins when installation is complete and no jobs are running” (see below). Installation of the plugin will be completed once Jenkins restarts.

restart jenkins

Installing the Plot Jenkins plugin (this is only necessary if you want to plot graphs in Jenkins)

In Jenkins, go to Jenkins -> Manage Jenkins -> Manage Plugins:

manage jenkins

Then click on the Available tab:

available

And search for Plot:

search plot

Click the “Plot” checkbox and then click the “Install without restart” button if you’re running jobs, or “Download now and install after restart” if you’re not running jobs:

plot checkbox

If you clicked the “Install without restart” button, then check the box “Restart Jenkins when installation is complete and no jobs are running” (see below). Installation of the plugin will be completed once Jenkins restarts.

restart jenkins

Installing the TextFinder Jenkins plugin (this is only necessary if you want to find text in the job log output and fail the job based on a text present in the job log)

In Jenkins, go to Jenkins -> Manage Jenkins -> Manage Plugins:

manage jenkins

Then click on the Available tab:

available

And search for TextFinder:

search textfinder

Click the “TextFinder” checkbox and then click the “Install without restart” button if you’re running jobs, or “Download now and install after restart” if you’re not running jobs:

textfinder checkbox

If you clicked the “Install without restart” button, then check the box “Restart Jenkins when installation is complete and no jobs are running” (see below). Installation of the plugin will be completed once Jenkins restarts.

restart jenkins

Adding the CloudTest Manager and CloudTest credentials in Jenkins

Now that we have all the plugins, we’ll create credentials in Jenkins for CloudTest Manager and CloudTest. Specifically, here’s what we’ll set up and what we’ll get:

  • The CloudTest Manager credentials will have privileges to access the CloudTest Environment.
  • The CloudTest credentials will have privileges to bring up the grid(s) we want to bring up.

First, let’s add the CloudTest Manager credentials. Go to Jenkins -> Manage Jenkins -> Configure System:

configure system

Then go to the CloudTest Servers section and click Add:

cloudtest servers add

In the next dialog box, specify a name in the Name field (this name is for internal Jenkins purposes and will be used when setting up a job, so be as specific as possible) and then enter the URL as https://cloudtestmanager.soasta.com/concerto/. This URL gives you access to CloudTest Manager. Finally, enter your CloudTest Manager user name and password. Once that’s entered, click Save.

username passwordsave

Finally, click the “Test Connection” button and you should see the “Success!” message:

success message

Next, let’s add the CloudTest credentials. Go to Jenkins -> Manage Jenkins -> Configure System:

configure system

Then go to the CloudTest Servers section and click Add:

cloudtest servers add

In the next dialog box, specify a name, similar to the other credentials be as specific as possible as this is something we’ll use in Jenkins when setting up a job, then enter the URL as your CloudTest URL (e.g., http://yourCloudTestServer/concerto/ — make sure that /concerto is in the URL path as shown), this URL is specific to your CloudTest environment and should be static. Then enter your CloudTest user name and password. Once that’s entered, click Save.

username password 2save

Finally, click the “Test Connection” button and you should see the “Success!” message:

success message

Creating a job in Jenkins for CloudTest

This section will show a sample Jenkins job configuration with CloudTest. Depending on your needs, you can skip certain segments (plotting in Jenkins, for example) if it’s something you don’t need.

To create a new job for CloudTest go to Jenkins -> New Item:

new item

Give a name to the job, click on “Freestyle project” and click OK.

freestyle

If you want to schedule the job to run periodically, check the “Build periodically” box under “Build Triggers” and then insert the desired scheduling (click the question mark on the right side of the text box to see examples/options).

build triggers

Now that we’ve set up the job, we’re ready for the next stage. Under “Build”, click “Add build step” and select “Start Test Environment”:

start test environment

Select the appropriate CloudTest Manager account from the drop-down (i.e., the account you added in the “Adding the CloudTest Manager and CloudTest credentials in Jenkins” section above) and insert the name of the CloudTest environment exactly as it appears in CloudTest Manager:

name server

Let’s now add the step to bring up the grid. Click again on “Add build step”, but this time select “Start Grid”, then select the CloudTest credentials that you added earlier and enter the path to the grid you want to bring up in the “Grid Name” box. The path is the location of the grid in CloudTest.

add build step

Let’s now add the next step, which is to play your composition. Click “Add build step” and “Play Composition(s)”, and then specify the same CloudTest Server credentials as you did for starting the grid step and enter the path to the composition you want to run. The path is the location of the composition in CloudTest.

play test composition

Let’s now work on the Post-build Actions. The following two steps (“Jenkins Text Finder” and “Plot build data”) are optional but helpful.

1. Jenkins Text Finder: Go to “Post-build Actions”, click “Add post-build action” and select “Jenkins Text Finder”.

jenkins text finder

Set it up as below, starting with “Files” as **/*.xml. Check the box “Also search the console output”, type “validation-fail” in the Regular expression field, and check the box “Unstable if found”. This will make sure that if CloudTest returns validation-fail, the build will be set as unstable.

post build actions

2. Plot build data: Go to “Post-build Actions”, click “Add post-build action” and select “Plot build data”. This will allow you to plot data from the test and see it in Jenkins without having the need to go to CloudTest.

plot build data

Enter the Plot group as “Response Time” and Plot title as “Transaction Average Response Time”, then set the number of builds to include to 10 and the Plot y-axis label as “Response Time”.

response time

In the “Data series file” field, type “AverageDuration.csv”. Then, select the radio button “Load data from csv file” and check the box “Display original csv above plot”:

average duration

Add another plot by clicking the “Add” button. This time, type the Plot group as “Response Time” and Plot title as “Transaction 90th Percentile Response Time”. Set the number of builds to 10 and Plot y-axis label as ”90th Percentile Response Time”:

90th percentile response time

In the “Data series file” field, type “90thPercentile.csv”. Then, select the radio button “Load data from csv file” and check the box “Display original csv above plot”:

90th percentile csv

Once the test is done, the last part of this setup is to bring down the grid and bring down the CloudTest environment.

To bring down the grid, click “Add post-build action” and select “Stop Grid”:

stop grid

Then select your CloudTest Server credentials and enter the path to the grid you want to bring down in the “Grid Name” field. The path is the location of the grid in CloudTest.

stop grid entry

Finally, to bring down the CloudTest environment, click “Add post-build action” and select “Stop Test Environment”:

stop test environment

Next, select the CloudTest Manager account from the drop down in the CloudTest Server field. In the Test Environment Name field, enter the exact name of your CloudTest environment as it appears in CloudTest Manager.

cloudtest name

Conclusion

You’ve now walked through the process to have a Jenkins job that will bring up the CloudTest environment, bring up the necessary grid(s), play your composition, plot results in Jenkins, and bring down grid(s) and environment.

If you set up the job to run at certain intervals, Jenkins will plot the data and you’ll see something like the samples below under the plots of the job once several builds have been run.

data plot sample #1
Jenkins plotted data, sample #1: Transaction 90th Percentile Response Time

 

data plot sample #2
Jenkins plotted data, sample #2: Transaction Average Response Time

 

Javier Garza is a developer evangelist at Akamai Technologies.