Load test tool

Simple but powerful command line tool for load testing your Java modules. It has a simple API that allows you to define testing tasks. It has a concept similar to JUnit/TestNG tests but with concentration on performance measurements in concurrent environment.

Installing

If you use a Debian based OS (Ubuntu, for example), you could simply download a .deb package from: http://github.com/bazhenov/load-test-tool/downloads

Then you should execute:

# dpkg -i load-test-tool-1.0-3.deb

Building

You need nothing more but maven for building load test tool.

$ mvn package

After building you will have .tar.gz artifact in ldt-assembly/target directory. You should unpack it in some directory where ldt will be installed:

$ tar xvf ldt-assembly-1.0-SNAPSHOT.tar.gz
x ldt-assembly-1.0-SNAPSHOT/bin/ldt.sh
x ldt-assembly-1.0-SNAPSHOT/bin/install.sh
x ldt-assembly-1.0-SNAPSHOT/lib/ldt-core-1.0-SNAPSHOT.jar
x ldt-assembly-1.0-SNAPSHOT/lib/commons-beanutils-1.7.0.jar
x ldt-assembly-1.0-SNAPSHOT/lib/commons-logging-1.0.3.jar
x ldt-assembly-1.0-SNAPSHOT/lib/commons-cli-1.1.jar
x ldt-assembly-1.0-SNAPSHOT/lib/log4j-1.2.14.jar
x ldt-assembly-1.0-SNAPSHOT/lib/ldt-assembly-1.0-SNAPSHOT.jar
$ alias ldt="~/Developer/ldt-assembly-1.0-SNAPSHOT/bin/ldt.sh"
$ ldt
usage: ltd
 -c,--concurrency-level <arg>   concurrency level
 -n,--count <arg>               sample count
 -p,--parameters <arg>          task parameters
 -r,--result-printer <arg>      result printer type (plain, log)
 -t,--timeframe <arg>           timeframe testing range (in milliseconds)
 -w,--warmup-threshold <arg>    warmup test execution count
 -z,--clazz <arg>               full qualified test class name
$ ldt -z com.farpost.ldt.Sleep -c 2 -n 10
                     RESULTS                      
--------------------------------------------------
 Concurrency level             : 2
 Samples count (per thread)    : 10
 Total time                    : 3.5s
 Min. time                     : 300ms
 Max. time                     : 304ms
 Std. dev.                     : ~349mcs
 Throughput                    : 6.7 tps

You could use alias or some other way to publish ldt.sh script in your shell environment (ln -s is a good way to start).

Task running

ldt can run tests written as a simple pojo objects. Let's see a sample:

public class MathTest {

	public void execute() {
		Math.sin(30);
	}
}

Method execute() called when you pass MathTest as a class name.

$ ldt -z org.mypackage.MathTest

This command will run tests and display execution statistic. It's worth to mention that ldt automatically adds current directory to classpath.

You could adjust concurrency level and thread sample count (task repeat count)

$ ldt -z full.quialified.ClassName -c 2 -n 100

In this example test defined in a class will be run in two separate threads 100 times in each thread.

If test class contains several tests (cases) you could choose which to run with following syntax:

$ ldt -z full.quialified.ClassName#test1

Strictly speaking test address full.quialified.ClassName equals to full.quialified.ClassName#execute.

ldt distribution contains several standard tasks. For example HttpTask for testing throughput of http services:

$ ldt -z com.farpost.ldt.HttpTask#get -p "url=http://google.com/" -n 10 -r log
1448
1011
997
987
989
2349
1229
990
983
1948

As you see I have not a very good link to google.com. I'm inclined to think that my physical location is reason to that ;) Anyhow, this example shows how you can use standard task in conjunction with different result printer (-r log) to get raw elapsed time log for several http requests.

Fixture

If you have some bootstraping code which must be executed before tests run you could define prepare() method. This method will be executed before any test in suite. All methods are called at one object instance, so prepare() method could communicate with test method using class properties:

public class SomeTest {
	
	private int[] numbers;

	public void prepare() {
		numbers = new int[1000];
	}

	public void test() {
		for ( int i : numbers ) {
			// ...
		}
	}
}

You could also define cleanup() method which will be executed only once after all test measurements are finished.

Warm up threshold

There are some reasons why in complex environments such as JVM you could want to skip first several test executions statistic (do not include it in resulting statistic). At the moment when JVM is started and run yor code, there are a lot activity going on. Some of them can influence your code hot paths (for example HotSpot JIT compiler). So ldt allows you to execute test several times before starting real measurements:

$ ldt -z full.qualified.test.name -w 10 -n 100 -c 4

In this example we run test 110 times in each thread, but ten first executions of each thread will be droped out and do not influence history.

Timeframe based testing

Instead of configuring test execution samples count you could just specifiy how long test should be executed (eg. 2 seconds). This is very convinient way of testing code especially when you do not know code throughtput in first place.

$ ldt -z com.farpost.ldt.HttpTask#get -p "url=http://baza.farpost.ru/" -t 2000
                     RESULTS                      
--------------------------------------------------
 Concurrency level             : 1
 Samples count                 : 6
 Total time                    : 2.139s
 Min. time                     : 289ms
 Max. time                     : 499ms
 Throughput                    : 2.8 tps

In this example ldt repeat test execution until cummulative execution time overgrown 2 seconds.

Test parameters

Sometimes there is need to pass some parameters to a test. Suppose following test example:

public class HttpRequestTest {

	private String url;
	private long timeout;

	public void setUrl(String url) {
		this.url = url;
	}

	public void setReadTimeout(long timeout) {
		this.timeout = timeout;
	}

	public void execute() {
		// reading url contents
	}
}

For passing url value in test we could use following command (parameters should be separated with comma):

$ ldt -z org.mypackage.HttpRequestTest -p "url=http://host.com, timeout=3000"

JVM startup options

In some cases you want to pass additional parameters to JVM when starting test (eg. heap size):

$ JAVA_OPTS="Xmx512M" ldt -z path.to.Test

Maven integration

ldt have full maven integration via maven plugin. At the moment you should build maven plugin by itself. Using maven this is as simply as:

$ mvn install

ldt have no heavy dependency list, so you shouldn't have any problems.

Then you should configure plugin in your pom.xml:

<project>

	<build>
		<plugins>
			<plugin>
				<groupId>com.farpost.maven.plugins</groupId>
				<artifactId>maven-ldt-plugin</artifactId>
				<configuration>
					<testName>com.farpost.test.ExampleTest</testName>
					<concurrencyLevel>3</concurrencyLevel>
					<callCount>1000</callCount>
				</configuration>
			</plugin>
		</plugins>
</project>

In order to use timeframe based testing you should set testTime parameter: <testName>com.farpost.test.ExampleTest</testName> <concurrencyLevel>3</concurrencyLevel> <testTime>2000</testTime>

At this point you should be able to run:

$ mvn ldt:test

and view test results

You also can pass ldt parameters from console line instead of pom.xml:

$ mvn ldt:test -Dldt.testName=com.farpost.test.ExampleTest -Dldt.concurrencyLevel=10 -Dldt.callCount=1000

License

Copyright (c) 2010 Denis Bazhenov

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.