Client/Server tests

The first issue to tackle is the lifecycle of the server tests, since it will need to break from the default behavior of JUnit’s lifecycle management.

To start, need a way to tell the server to start up and initialize. This should be done using setUp() method. Once this method returns, it is assumed that the server is ready to receive calls. Next, need a way to tell the server that it can shutdown and clean up. This should be done using the tearDown() method. This method will only be called in the case that all clients are finished making their calls to the server.

This behavior is all provided by having the server test case class extend org.jboss.jrunit.ServerTestCase instead of junit.framework.TestCase. The ServerTestCase actually extends the TestCase class itself, but overrides a few it’s methods to get the desired behavior.

The server implementation may also have a test method, which will be called just after the setUp(), as in the case of regular JUnit test runs. This test method can contain asserts and are suggested to be used for validation of server data and metrics.

A few important points… there can be only ONE test method within a server test case. This is because JUnit creates a new instance of the test class for each test method run. Therefore, if had multiple tests, would be multiple instances of the server test case created. To change this would require a sizable change to the JUnit code, so please just use one test method (or contribute the required change  ).

Another important point is that the tearDown() method may be called even while a test method is being run. This is intentional and allows for the test method to loop until the tearDown() method is called. So a possible example of where this could be used is:

...
   public void testServerMetrics()
   {
      while(!stop)
      {
         // collect data here
      }
   }

   protected void tearDown()
   {
      stop = true;
      // so will cause testServerMetrics() to break out of loop 
      // do shutdown and clean up code. 
   }
...

For the client test case, there is no requirement for jrunit other than they extend the junit.framework.TestCase class and conform to normal constraints of a JUnit test case.

Lastly, there is the org.jboss.jrunit.TestDriver class, which represents the driver for the client and server tests. This class will spawn new test harnesses that the client and server tests cases will run within. The test driver will then communicate to the test harnesses using a JGroups message bus to control the test lifecycle for the server test case and all the client tests cases as well as obtaining the results from those test runs.

The logical order of a test run as controlled by the test runner is:

The only user coding required for the test driver is to implement a class that extends the org.jboss.jrunit.TestDriver abstract class and implement the declareTestClasses() method. Within this method, call the TestDriver’s addTestClasses() method and specify the client test case class, the number of clients to spawn, and the server test case class.

Benchmark Decorators

JRunit can also utilize the benchmark decorators to provide benchmark results as well as regular JUnit test results.

org.jboss.jrunit.decorators.ThreadLocalDecorator (and all the other decorators that accept number of threads and loops) - for each thread specified in the number of threads, there will be a new instance of the test created. For the number of loops specified, the test methods will be called on each test instance. So for example, if specify 3 threads and 10 loops, there will be three instances of the test class created and each instance will have their test methods called 10 times by each of their respective threads. An example class that demonstrates this would be:

public class SimpleThreadLoopCounter extends TestCase
{
   private static int staticCounter = 0;
   private static int staticMethodCounter = 0;
   private int localCounter = 0;

   public static Test suite()
   {
      return new ThreadLocalDecorator(SimpleThreadLoopCounter.class, 3, 10, 0, true, true);
   }

   public SimpleThreadLoopCounter()
   {
      staticCounter++;
   }

   public void testCounter() throws Exception
   {
      System.out.println("staticCounter = " + staticCounter);
      System.out.println("staticMethodcounter = " + ++staticMethodCounter);
      System.out.println("localCounter = " + ++localCounter);
   }
}

When this is run, the last entry will be:

staticCounter = 3 staticMethodcounter = 30 localCounter = 10

Another issue is the way junit treats test classes with multiple test methods. For each test method that junit runs within a test class, it will create a new test instance to run that test method. To illustrate this, if copied the testCounter() method from the example above (calling it testCounter2() for example), the last lines printed for the test run would be:

staticCounter = 6 staticMethodcounter = 60 localCounter = 10

Issues with jrunit and decorators: The ServerTestHarness will not work if the numberOfThreads is more than 1. I don't see this as being that big of a problem from the point of view that the numberOfThreads is to simulate multiple clients running at the same time. If want to do this using the ServerTestHarness, can actually spawn multiple clients through it. Eventhough they won't be running concurrently with the same processes, they will be running concurrently with seperate processes. Having the ability to use the loop parameter is needed though so can keep the clients calling on the server for an extended period of time (or iterations). Therefore, if running remote tests and want to use the ThreadLocalDecorator for benchmark data, use the following ThreadLocalDecorator constructor, which will default the number of threads to one:

public ThreadLocalDecorator(Class testClazz, int loops)