Installing and Running JMeter

You can download the latest stable version from JMeter's site. Downloads area available as either .gz or .zip files. JMeter 1.8 requires a working JDK 1.4 environment.

Once you extract the binary distribution file, JMeter is ready for you. On Linux/UNIX, run JMeter by invoking the jmeter shell script. On Windows, call the jmeter.bat file. Both files can be found in the bin/ directory of the JMeter installation directory. Figure 1 shows JMeter's main window, which is a Swing application.

Figure 1: JMeter's main window.

The user interface has two panes. The left pane displays the elements used in our testing. Initially, there are the Root and two sub-elements, Test Plan and WorkBench. In this article we're only concerned with Test Plans. Add an element to a node by right-clicking it and selecting Add. To remove an element, select the element by clicking on it, then right-click on the element and choose the Remove option.

The right pane of the user interface displays the details of each element. You are now ready to use JMeter. There are two things to note:

1. You should not run JMeter on the same machine running the application to be tested. JMeter may use extensive resources that might affect the other application's performance if they are both run on the same machine.
2. Make sure that the testing is affected as little as possible by network traffic. The best thing to do is to ask your network administrator to set up an isolated sub-network for the machine running the Web application and the machine running JMeter.

Using JMeter for a Simple Test

Let's start with a very simple test. In this test, we will set up a test plan and stress test a Web application. You will be introduced with some common concepts in JMeter. After understanding this basic test, you should be able to use all of the capabilities of JMeter.

To conduct a test, you must have a test plan. A test plan describes the steps that JMeter will take to perform the testing. A test plan includes elements such as thread groups, logic controllers, sample generating controllers, listeners, timers, assertions, and configuration elements. Don't worry at this stage if you don't understand what these elements are.

A test plan must have at least one thread group. A thread group is the starting point of a test plan, and it can contain all other JMeter elements. A thread group controls the threads that will be created by JMeter to simulate simultaneous users.

Now, let's start by creating a thread group. Right-click the Test Plan element and select Add and then Thread Group. JMeter will create a thread group element under Test Plan element. Click the Thread Group element, and you will see a screen like the one in Figure 2.

Figure 2: Configuring a thread group.

In this page, you can set the following properties:

• Name -- the name of this thread group. You can give a descriptive name to this property.
• Number of Threads -- the number of threads created. Each thread represents a single user. Therefore, if you want to simulate a load test with 10 concurrent users, enter 10 as the value for this property.
• Ramp-Up Period -- the number of seconds JMeter will take to accelerate to create all of the threads needed. If the number of threads used is 10 and the ramp-up period is 20 seconds, JMeter will take 20 seconds to create those 10 threads, creating one new thread every two seconds. If you want all threads to be created at once, put 0 in this box.
• Forever -- if clicked, this option tells JMeter to keep sending requests to the tested application indefinitely. If disabled, JMeter will repeat the test for the number of times entered in the Loop Count box.
• Loop Count --this property value only has an effect if the Forever check box is unchecked. It tells JMeter the number of times it has to repeat the test.

For our simple test, fill the properties with the values found in Figure 2. We will use two users and each test will be performed three times. We use small numbers here so that we can easily explain the results later; in real load testing, you might want to use higher numbers for these properties.

Next, you need to add the element that represents HTTP requests. To do so, right-click the Thread Group element, and select Add, Sampler, and then HTTP Request. An HTTP Request element will be added to the Thread Group element. Click the HTTP Request element to select it, and you should see a screen similar to Figure 3.

Figure 3: Configuring an HTTP Request element.

On the HTTP Request screen, you configure the HTTP requests that will be used to "hit" your application. Here, you can set the following properties.

• Name -- the name of this HTTP request. The name should be descriptive; remember that it is common to have multiple HTTP Request elements in a thread group.
• Server Name or IP -- the server name or the IP address of the machine running the application being tested.
• Port Number -- the port number used by the application. Normally, a Web application runs on port 80.
• Protocol -- the protocol used, either HTTP or HTTPS.
• Method -- the request method, either GET or POST.
• Path -- the path to the resource that will handle this request.
• Follow Redirects -- follows redirections sent by the Web application, if any.
• Use KeepAlive -- if checked, sends the Connection = Keep-Alive request header. By default, an HTTP 1.1 browser uses Keep-Alive as the value of the Connection header. Therefore, this checkbox should be checked.
• Parameters -- the list of parameters sent with this request. Use the Add and Delete buttons to add and remove parameters.
• Send a file with a request -- simulate a file upload to the Web application.
• Retrieve all images and Java Applets -- download embedded content.

Now you should be able to figure out the values for the properties in the HTTP Request page. The last element that we need to add to our test plan is a listener, which in JMeter is the same as a report. JMeter comes with various reports to choose from. A report can be a table or a graph. For this testing, use the easiest report available: a table.

To add a listener, right-click the thread Group element, select Add, and then Listener and View Results in Table. Now you are ready to run the test plan. Before you run your test plan, however, you are advised to save the test plan just in case JMeter crashes the system (an occasional occurrence with higher numbers of threads and loop counts). Afterwards, select Start from the Run menu to execute the test plan.

While the test plan executes, the small box on the bar right below the menu bar will turn green. For a test that does not run indefinitely, JMeter will automatically stop the test plan after it's finished. For a test that goes on indefinitely, you must intervene to stop the test. Do this by selecting Stop from the Run menu.

When I ran my test plan, I got the results like those shown in Figure 4.

Figure 4: A table report.

It is very easy to understand the figures in the table. There are six samples taken (two threads and three loop counts, thus 2 x 3 = 6). The response time from each sample is given in the third column. They are 100, 60, 260, 50, 120, and 80 ms. All samples are taken successfully, as described by the fourth column. On average, each sample has a response time of 111 ms ((100 + 60 + 260 + 50 + 120 + 80)/6).

Another important figure is the standard deviation, defined as the square root of the total of the deviation of each sample from the average. This figure indicates how stable your Web application is. If the standard deviation is high, some users will experience very good responses while some other users will wait for a longer time. The smaller this value, the better.

After conducting a simple test, it is very easy to do more complex tests. For load testing Web applications, increase the number of threads and the loop counts gradually and see how your applications cope with the loads. The following sections of this article tackle some other important aspects of load-testing Web applications with JMeter.

Listeners

JMeter comes with a number of listeners or reports. In the previous test, we used a table to display the test results. If this is not suitable for you, you can choose one or more of the other listeners for a thread group. A popular listener is the Graph Results, as shown in Figure 5.

Figure 5: Graph results.

Multiple HTTP Requests

A real application has multiple resources, both static and dynamic. Chances are, you want to see the performance of these resources. JMeter makes it easy to employ multiple HTTP requests. Just add any number of HTTP Request elements and configure them, as in the previous test. If you have multiple HTTP Request elements, you might want to use a HTTP Request Defaults element, described in the following section.

HTTP Request Defaults

The HTTP Request Defaults element specifies the default values of existing HTTP Request elements within the same thread group. The HTTP Request Defaults element is especially useful because most, if not all, HTTP Request elements normally have the same server and port. Figure 6 shows the detail page of a HTTP Request Defaults element.

Figure 6: HTTP request defaults.

Add an HTTP Request Default element by right-clicking a Thread Group element and then selecting Add, Config Element, and HTTP Request Default.

Cookie Manager

Many Web applications use cookies. JMeter provides cookie capabilities through a Cookie Manager. Adding this element to a thread group allows you to send cookies to the application being tested, just as Web browsers do. Figure 7 displays the details page of a Cookie Manager. Here you can add and delete a cookie.

Figure 7; Cookie Manager.

You can add a Cookie Manager element by right-clicking a Thread Group element and then selecting Add, Config Element, and Cookie Manager.

Summary

JMeter is capable of much more than our simple tests demonstrate. From these building blocks, it's possible to create extensive tests with highly detailed reports. Getting useful results is very easy, though.

Running the Test Plan

To run the test plan you can go to Run menu -> Run or press ctrl + r. To indicate that the test is running a green square lits up in the upper-right-hand corner. After selecting "stop" as well the green light remains on unless and until all test threads exit. Furthermore the total number of currently active threads is indicated by the numbers to the left of the green square. The green square will change to grey Once JMeter  finishes running your Test Plan. You can also stop the test plan in between by selecting Shutdown or Stop from the Run menu.

A file which you want to save the results for in your listener could be opened in any visualizer. The results will be displayed according to the visualizer itself. However you can open the same file in more than one visualizer because during the test run the JMeter will ensure that no sample is recorded more than once to the same file.

Configuring JMeter

The properties of JMeter can be modified by implementing changes in the jmeter.properties in the /bin directory or you can also develop a copy of jmeter.properties on your own by specifying it in the command line.

Some more JMeter properties can be defined in the file as defined by the JMeter propertyuser.properties having the default value user.properties. If the file is in the current directory then it will automatically get loaded. In the same way you can update the system properties fromsystem.properties.
Parameters

Attribute	Description	Required
remote_hosts	Comma-delimited list of remote JMeter hosts. You need to list the         machines where you have JMeter remote servers running because then you will be able to control those servers in a distributed environment from this machine's GUI.	NO
not_in_menu	With the help of this you can customize your JMeter by allowing only those components to display which you want to by listing their classname or their class label.	NO
user.properties	Addition JMeter properties are contained in this file and are added before the -q and -J options are processed and after the initial property file.	NO
xml.parser	The default value is: org.apache.xerces.parsers.SAXParser. An implementation can be specified as XML parser.	NO
search_paths	This displays the list of paths to search JMeter add-on classes by JMeter such as additional samplers which is in addition to the jars kept in  lib/ext directory.	NO
system.properties	This contains additional system properties which are added before the -S and -D options are processed.	NO
user.classpath	A path to be searched for utility classes by JMeter in addition to jars kept in lib directory.	NO
ssl.provider	If you don't require the built-in Java implementation then the class can be specified by you for your SSL implementation.	NO

Enhancing the Test Plan

As you have seen we have added some basic elements to the Test Plan like Sampler and some listeners. Similarly you can add many other elements in your Test Plan as shown under Group heading.

Group	Purpose
Samplers	The requests to the servers are being sent by these elements for the request types SOAP, JDBC,"Java", HTTP/HTTPS, FTP,
Listeners	The results of the run can be saved by these elements in the single request, tabular forms etc by saving the results in a named file.
Timers	To make the test more realistic we use Timers element to insert delays between the requests.
Logic	Logic controllers are used if you have defined the request to be executed on some logic like if-then-else and loop structures in Java etc.
Configuration Elements	common information about the requests is added to work with samplers using this element.
Assertions	This element is used to check whether you are getting the responses within a given amount of time containing the expected data or not.

What does Apache JMeter do?

• Loads and does the performance test for HTTP and FTP servers and arbitrary database queries.
• Does the Caching and offline analysis of test results.
• concurrent sampling by many threads as well as sampling of different functions by seperate thread groups simultaneously.
• Its extensibility includes:      
                      supports Scriptable Samplers such as BeanShell and BSF-compatible languages.
                      More extendibility and personalization allowed by Data analysis and visualization.
                      Unlimited testing capabilities by Pluggable Samplers.
• Complete portability
• Faster operation and more precise timings is done by GUI design. 
• Supports Full Swing and lightweight component. 

JMeter is not a browser

Well, don't take it as a browser at all because Jmeter does not perform the actions supported by browsers as far as web-services and remote services are concerned. No Javascript is executed by the JMeter found in HTML pages. However it sometimes happens that the response could be viewed as HTML etc excluding the timings in any samples. Remember, you can view only one sample in one thread at a time.

Error reporting

Errors and warnings are also depicted by the JMeter. These errors and warnings are reported to thejmeter.log file by the JMeter including some information which runs itself on the test. Some errors which are not depicted by the JMeter may appear on the command console. You need to check the log file for any errors being reported if in case the test performed by you is not going the way to be expected. For instance a syntax error in a function call might be reported in the log file.

The errors which are not trapped and logged by the JMeter get stored as attributes of the sample result. For instance sampling errors like HTTP 404 - file not found etc get stored as attributes of the sample result.

Running the load test

You can also view the graph for your test after running it. For this go to Thread group -> Add -> Listener -> Graph results as shown in the picture below. After clicking on the graph element, you will see the chart getting populated with the data representing requests to the server.  

The chart  above shows the Test results with number of threads = 5, ramp-up time = 10, loop count = 100, and loop controller set to 1.

Three parameters on the charts have been displayed - throughput (green line), median (purple line) and average (blue line). If we modify the parameters to simulate a higher load on a server by increasing the number of threads to 50, and changing the ramp-up time to 5 and loop count to 20, then we get the following graph by clicking the start again.

The meaning of the parameters at the bottom of the chart is as follows: 

• The number of requests processed by the server per minute is denoted by Throughput.
• Average is being calculated as the total time running divided by number of requests sent to the server.
• Time is represented by Median i.e. server's half response time is lower than this number.
• The variation in the server's response time is denoted by Deviation.
• The completion of any last request is shown by Latest.

The summary of the above two runs is shown below in the form of a table.

Number of threads	Throughput, responses/minute	Average, ms	Median, ms
5	1665	57	31
50	1749	314	62

You can also add the Spline Visualizer to further visualize the test results of all sample times. It provides a smooth view of distribution based on polynomial approximations as a continuous line. The picture below displays the screen of Spline Visualizer.

The same test results have been shown using Spline Visualizer.

Using JMeter for a Simple Test

     

Lets see how to run JMeter now. We will conduct a simple test to set up a test plan and stress test a Web application. Before proceeding with the test we need to have a test plan first which will help the Jmeter to perform the testing in steps. There are several elements in a test plan like thread groups, listeners, assertions, sample generating controllers, logic controllers etc. Well, these elements will be described later. One point to remember here is that a test plan must have at least one thread group having all other JMeter elements which is the starting point of a test plan. All of the other threads created by the Jmeter to simulate simultaneous users will be controlled by this thread group. Lets go through the steps now.
Step 1: Start JMeter by running the JMeter.bat file for Windows or the JMeter file on Unix.

Step 2: Create a thread group by right-clicking the Test plan element as shown in the picture below. Now select ADD and then the Thread group option. After selecting the thread group option a thread group element will be created by the Jmeter under Test Plan element. This is to confirm about the number of users to be stimulated by you and the number of times the test plan to be repeated. 

The screen after creating the Thread group appears like this.

Following properties can be set as shown below:
Name -- You can give any name to the thread group.                                               Number of Threads -- You can enter as many threads to stimulate a load test. A single user is represented by each Thread so if you wish to simulate a load test with 5 concurrent users then you need to enter 5 as the value for this property.                 
Ramp-Up Period -- It indicates the time taken by Jmeter to create all of the threads needed. If you set 10 seconds as the ramp-up period for 5 threads then the JMeter will take 10 seconds to create those 5 threads. Also by setting its value to 0 all the threads can be created at once.                                                                 
Forever -- If you choose this option then the Jmeter will keep sending the unspecific requests to the tested application. And if disabled then the test will be repeated the number of times entered in the Loop Count box.                                                                                                           
Loop Count -- By specifying its value Jmeter gets to know that how many times a test is to be repeated provided that the Forever check box should be unchecked.                                 
Step 3: Now you need to mention the HTTP request (URL and parameters). To this right click on the Thread Group node then you need to select Add -> Sampler -> HTTP Request as shown in the picture. 

The picture below shows the screen of HTTP Request in which you can set some properties as described below. Here address which we have taken for the "Hello World" servlet ishttp://localhost:8080/examples/servlets/servlet/HelloWorldExample.

The following properties can be set on the HTTP Request screen.                        
Name -- You should put a descriptive name as a thread group can have multiple HTTP Request elements.                                                    Server Name or IP -- Mention the server name or the IP address of the machine which is running the application to be tested.
Port Number -- Give the port number on which the Web applications run which is usually 80.  Protocol -- Mention the protocol to be used here i.e. either HTTPS or HTTP.Method -- The GET or POST method is to be mentioned here.                 Path -- You need to mention the path of the resource that will handle this request.                       Follow Redirects -- follows redirections, if any, sent by the Web application.Parameters -- This option shows the list of parameters sent with the request. You can add and remove parameters by using Add and Delete buttons.
Send a file with a request -- With the help of this option a file upload can be simulated to the Web application.
Retrieve all images and Java Applets -- This option is used to download embedded content.
Step 4: Now the format of the results is to be selected to get a page containing the results of every request by right-clicking on the Thread group node then select Add -> Listener -> View Results Tree.

Step 5: Now its time to run the Test plan by selecting Run from the menu and then select Start (or Ctrl-R): Hence The test plan will be repeated 10 times in View Result Tree as shown below.

Step 6: The results can been seen in the Results Tree after the completion of the Test plan. As you can seen in the picture below we have selected the first request in the upper pane and also the request which was generated with the results in the form of an HTML page with the text "Hello World!"

Step 7:  JMeter has got an interesting feature which is Add more listeners. You can add a View Results in Table listener to view the requests in tabular form as shown below.

Moreover you can add an Aggregate Report listener to get the summary of run as shown below.   

                                                                                    
Step 8: To save the test plan for later use select File from the menu and then Save Test Plan (or Ctrl-S). Its preferable to save the test plan before running it.

What is JMetter?

As the name indicates Jmeter is one of the Java tools which is used to load testing client/server applications. Earlier it was used for testing Web Applications only however now-a-days its being used for other test functions. It is typically used to measure performance and to load test functional behavior of client-server applications. It can also handle FTP and JDBC requests. You must be aware of such other tools like WinRunner. You will be glad to know that as compared to WinRunner, Jmeter is easy to use due to its simple and intuitive GUI. Moreover its absolutely free and can be modified at ease as its an open source. 

The important functionalities of Jmeter is that a heavy load on a server can be stimulated by using it, not on a server but also a heavy load on a network or object to test its strength under different load types. Agraphical analysis of performance can also be done by using Jmeter or the behaviour of your server/script/object can also be tested under heavy concurrent load.

Before going any further lets see how to install it.

Jmeter Installation

The latest version of Jmeter is 1.8 which you can download from JMeter's site, once you download the JMeter zip file you need to unzip it. Jmeter is compatible with JDK 1.4 environment and it can be downloaded as .gz or .zip files. To run JMeter you need to extract the binary distribution file. Jmeter can be run on different OS as described below:

• You need to invoke the jmeter shell script on Linux/UNIX to run JMeter.
• If you want to run Jmeter on WINDOWS then call jmeter.bat file. You can find both the files in the bin/ directory of the JMeter installation directory. 

The JMeter's main window is displayed below which is a Swing application.

As shown in the figure above, there are two panes of the user interface. The elements used in our testing is shown by the left pane. Here we are only concerned with the Test Plans as there were two sub-elements earlier that are Test Plan and WorkBench. Furthermore an element can be added to a node on right-clicking and can be removed also by selecting the element and then choosing the remove option by right-clicking on it. Whereas the details of each of the elements is shown by the right pane. 

There are two things to remember before using the JMeter:

1. If you want to test any application then make sure that you run JMeter on the other machine because it might be possible that Jmeter could affect the other application's performance running on same machine due to the use of some extensive resources.
2. It is always recommendable to set up an isolated sub-network for the two machines running Web application and the JMeter in order to not to affect the testing by network traffic. 

JMeter's Classpath

The classes from jars are automatically found by the Jmeter in the following directories:

 JMETER_HOME/lib - used for utility jars
 JMETER_HOME/lib/ext - used for JMeter components 

One point to remember here is that you should jar any JMeter components developed by you and also you need to copy the jar into JMeter's lib/ext directory. JMeter components will be automatically found by the JMeter in any jars found here.

Furthermore you should define the property search_ paths in jmeter.properties in case you don't want to put the extension jars in the lib/ext directory. And also lib/ext is intended for JMeter componentsonly and not for utility jars so don't use lib/ext for utility jars. Remember that other jars should be placed in the lib directory - not the lib/ext directory, jars like, JavaMail, JDBC, Beanshell etc.

Note: Only .jar files are found by JMeter not the .zip files.

JMeter starts with "java -jar" so setting the CLASSPATH environment variable will be of no effect as the CLASSPATH variable is ignored by the java command because it happens with all Java programs, not just JMeter.