This covers the samples include within the JRunit project. The main focus of the samples covered here are related to client/server tests. All the samples illustrate the different features of microbenchmark and jrunit using the same base client and server base code which uses a socket to send a String to the server from the client, which is then returned a static String from the server. All the samples can be compiled and run within an IDE, but some can be run via the ant build script included in the root directory of jrunit. The reason that only some are included within the ant script is that they can be completely automated (which will be covered below). Also note that each section title for the samples below corresponds to the org.jboss.jrunit.sample package that they can be found under.
The first sample for client/server testing is in the org.jboss.jrunit.sample.basic package, which contains SimpleServerTest and SimpleServerTest. These are the tests that we will build on for the rest of the test cases that demonstrate the different features. The SimpleClientTest is a basic JUnit test case with two test methods, testClientCall() and testFailure().
public class SimpleClientTest extends TestCase
{
private String address = "localhost";
private int port = 6700;
private Socket socket = null;
private ObjectOutputStream oos;
private ObjectInputStream objInputStream;
public void testClientCall() throws Exception
{
getSocket();
oos.writeObject("This is the request from " + Thread.currentThread().getName());
oos.reset();
oos.writeObject(Boolean.TRUE);
oos.flush();
oos.reset();
Object obj = objInputStream.readObject();
objInputStream.readObject();
assertEquals("This is response.", obj);
}
public void testFailure()
{
fail("This is a check to make sure failures reported.");
}
private void getSocket() throws IOException
{
if(socket == null)
{
try
{
socket = new Socket(address, port);
BufferedOutputStream out = new BufferedOutputStream(socket.getOutputStream());
BufferedInputStream in = new BufferedInputStream(socket.getInputStream());
oos = new ObjectOutputStream(out);
objInputStream = new ObjectInputStream(in);
}
catch(IOException e)
{
e.printStackTrace();
}
}
else
{
oos.reset();
oos.writeByte(1);
oos.flush();
oos.reset();
objInputStream.readByte();
}
}
}The testClientCall() method will open a socket to the server, send a String and receive a String in return and check that the returned String is the value it expected (using an JUnit assert). The testFailure() method will just fail a JUnit assertion.
The SimpleServerTest is a basic JUnit test case that sets up a socket server within it's setUp() method (on a separate thread) which will accept requests from clients and respond with a static String. The SimpleServerTest also has a test method, called testRequestCount(), which will simply print out the number of request its taken every 3 seconds.
public class SimpleServerTest extends TestCase //ServerTestCase
{
private int serverBindPort = 6700;
private int backlog = 200;
private ServerSocket serverSocket;
private InetAddress bindAddress;
private String serverBindAddress = "localhost";
private int timeout = 5000; // 5 seconds.
private boolean continueToRun = true; // flag to keep the server listening
private int requestCounter = 0;
protected void setUp() throws Exception
{
System.out.println("SimpleServerTest::setUp() called.");
bindAddress = InetAddress.getByName(serverBindAddress);
serverSocket = new ServerSocket(serverBindPort, backlog, bindAddress);
// this was done inline since TestCase already has a void parameter run() method
// so could not create a run() method for the Runnable implementation.
new Thread()
{
public void run()
{
try
{
startServer();
}
catch(Exception e)
{
e.printStackTrace();
}
}
}.start();
}
public void testRequestCount()
{
while(continueToRun)
{
try
{
Thread.currentThread().sleep(3000);
}
catch(InterruptedException e)
{
e.printStackTrace();
}
System.out.println("Requests taken: " + requestCounter);
}
assertEquals(30, requestCounter);
}
protected void tearDown() throws Exception
{
continueToRun = false;
System.out.println("Tearing down. Processed " + requestCounter + " requests");
if(serverSocket != null && !serverSocket.isClosed())
{
serverSocket.close();
serverSocket = null;
}
}
private void startServer()
{
while(continueToRun)
{
try
{
Socket socket = serverSocket.accept();
socket.setSoTimeout(timeout);
BufferedOutputStream bos = new BufferedOutputStream(socket.getOutputStream());
BufferedInputStream bis = new BufferedInputStream(socket.getInputStream());
ObjectOutputStream oos = new ObjectOutputStream(bos);
oos.flush();
ObjectInputStream objInputStream = new ObjectInputStream(bis);
processRequest(objInputStream, oos);
while(continueToRun)
{
acknowledge(objInputStream, oos);
processRequest(objInputStream, oos);
}
}
catch(IOException e)
{
System.out.println("Done processing on client socket.");
}
catch(ClassNotFoundException e)
{
}
}
}
...
This introduces one of the problems with JUnit testing from a client/server standpoint. The client test needs the server to be available to accept requests for the duration of all it's tests. The server, since it follows the conventional JUnit model, wants to set up, test, and then tear down. When the server and then the client tests are run, the testClientCall() method of SimpleClientTest will pass. However, the testRequestCount() will never end and the tearDown() method will never be called.
To stop the SimpleServerTest from running, the process will have to be killed. If the code is changed so the testRequestCount() method of SimpleServerTest does not loop, it is very possible that the test run of the SimpleServerTest would be finished before the SimpleClientTest would have a chance to call on it, thus causing the testClientCall() to fail.
Although this sample does not use the jrunit features directly, it does show the power of using the benchmark decorator. This sample uses a slight variation of the basic SimpleClientTest and SimpleServerTest.
public static Test suite() { return new ThreadLocalDecorator(SimpleClientTest.class, 3, 10, 0, true); } public void testClientCall() throws Exception { // start benchmark tracking ThreadLocalBenchmark.openBench("ClientCall"); ThreadLocalBenchmark.openBench("GetSocket"); getSocket(); ThreadLocalBenchmark.closeBench("GetSocket"); /** * Uncomment this to see that is the same object, thread, and socket for each loop, * but will be different for each thread as specified to the test decorator. This * is based on the values passed to JunitThreadDecorator (or one of it's subclasses) * for the numberOfThreads and loop parameter values. */ //System.out.println(hashCode() + " - " + Thread.currentThread().getName() + " - " + socket.hashCode()); oos.writeObject("This is the request from " + Thread.currentThread().getName()); oos.reset(); oos.writeObject(Boolean.TRUE); oos.flush(); oos.reset(); Object obj = objInputStream.readObject(); objInputStream.readObject(); assertEquals("This is response.", obj); ThreadLocalBenchmark.closeBench("ClientCall"); }
The SimpleClientTest was modified to declare a suite() method. This method will return a new instance of the org.jboss.jrunit.decorators.ThreadLocalDecorator class. In this particular case, the parameters specify that the decorator should run the test case org.jboss.jrunit.sample.decorated.SimpleClientTest, using 3 separate threads, each looping 10 times, with no delay (see ThreadLocalDecorator section for more information on configuring the ThreadLocalDecorator).
This means that when the SimpleClientTest is run, there will be three instances created that run all the tests 10 times each. The testClientCall() method of the SimpleClientTest class also has some lines added to mark the starting and ending of named benchmarks. In particular, the ClientCall and the GotSocket benchmark will be started at the beginning of the method.
The GotSocket benchmark will be stopped after making the call the getSocket() method and the ClientCall benchmark will be closed at the end of the method. At this point the SimpleServerTest has not modified. Therefore, based on the code, it will accept only one client socket connection at a time and process the requests for this connection. After that connection is closed, it will move onto the next connection.
To run this sample, start the org.jboss.jrunit.sample.basic.decorated.SimpleServerTest and then the org.jboss.jrunit.sample.decorated.SimpleClientTest. After the client has finished, stop the server and at the bottom of the client console, should see something similar to:
SubBenchmarks: Benchmark:ClientCall Executions:30 Time:15311 SubBenchmarks: Benchmark:GetSocket Executions:30 Time:15191
This indicates how long it took for the three threads to run ten iterations of the testClientCall() and getSocket() methods. Now, uncomment the for loop in the setUp() method of the SimpleServerTest class. This will make it so there are three threads available to accept incoming client socket request (please do not send me any complaints about how this is NOT how servers should be coded... it for demonstration purposes, relax :) ). Compile and re-run. You results should now look something like:
SubBenchmarks: Benchmark:ClientCall Executions:30 Time:130 SubBenchmarks: Benchmark:GetSocket Executions:30 Time:30
These results indicate that just by changing the server to be multi threaded so that it could accept and process all the client requests concurrently the client calls executed over 100 times faster.
The next example, which is under org.jboss.jrunit.sample.clientserver package addresses how jrunit helps to solve these issues when running a client/server test within a JUnit construct. The only change required to the original test classes is that the SimpleServerTest extends org.jboss.jrunit.ServerTestCase instead of junit.framework.TestCase, which will allow the behavior change needed within the server test run (see the section on ServerTestCase for more details on how this class changes the test run behavior).
public class SimpleServerTest extends ServerTestCase // NOTE: Not TestCase, but ServerTestCase
{
...Also changed the test to use log4j instead of System.out.println for logging since output will not be to console, more on why this is in a minute. The other class within the org.jboss.jrunit.sample.clientserver package is the SampleClientServerTest. This class extends the org.jboss.jrunit.harness.TestDriver class which provides the harness for running the client and server in different processes.
public class SampleClientServerTest extends TestDriver { public void declareTestClasses() { addTestClasses("org.jboss.jrunit.sample.basic.SimpleClientTest", 1, "org.jboss.jrunit.sample.clientserver.SimpleServerTest"); } protected Level getTestHarnessLogLevel() { return Level.DEBUG; } }
The main method of note is the declareTestClasses(), which must be implemented as is an abstract method within TestDriver. This method then calls the addTestClasses() method from the TestDriver. The parameters to the addTestClasses() method, in order, are the client test class to run, the number of clients processes to spawn, and the server test class to run. Notice that the client specified is the original org.jboss.jrunit.sample.basic.SimpleClientTest class and the new org.jboss.jrunit.sample.clientserver.SimpleServerTest, which extends ServerTestCase.
There is also the getTestHarnessLogLevel() method, which tells the driver what the log level should be set to for the test harness code, for debugging purposes. To run this sample, go to the root directory and run the ant target run-SampleClientServerTest. The console output should be similar to:
run-SampleClientServerTest: [junit] Running org.jboss.jrunit.sample.clientserver.SampleClientServerTest [junit] Tests run: 1, Failures: 2, Errors: 0, Time elapsed: 10.045 sec [junit] Test org.jboss.jrunit.sample.clientserver.SampleClientServerTest FAILED [junitreport] Transform time: 471ms
This ant task will also create an html report under the output/test-report/ directory. Reviewing the results should indicate that there were two failures. The first being the failure from the testFailure() method of the org.jboss.jrunit.sample.basic.SimpleClientTest class and the other being from the testRequestCount() method of the org.jboss.jrunit.sample.clientserver.SimpleServerTest (because only received one client call instead of 30).
This next sample is basically the as the previous sample except will run three client processes to call on the server instead just the one. The only new class for this example is the org.jboss.jrunit.sample.multipleclientserver.SampleMultipleClientServerTest.
public class SampleMultipleClientServerTest extends TestDriver
{
public void declareTestClasses()
{
addTestClasses("org.jboss.jrunit.sample.basic.SimpleClientTest",
3,
"org.jboss.jrunit.sample.clientserver.SimpleServerTest");
}
}The only difference in this class and the SampleClientServerTest from before is that change the second parameter to addTestClassses() method from 1 to 3. To run this sample, go to the root directory and run the ant target run-SampleMultipleClientServerTest. The console output should be similar to:
run-SampleMultipleClientServerTest: [junit] Running org.jboss.jrunit.sample.multipleclientserver.SampleMultipleClientServerTest [junit] Tests run: 1, Failures: 4, Errors: 0, Time elapsed: 11.577 sec [junit] Test org.jboss.jrunit.sample.multipleclientserver.SampleMultipleClientServerTest FAILED [junitreport] Transform time: 511ms
This ant task will also create an html report under the output/test-report/ directory. Reviewing the results should indicate that there were now four failures. The first three being the failure from the testFailure() method of the org.jboss.jrunit.sample.basic.SimpleClientTest class and the fourth being from the testRequestCount() method of the org.jboss.jrunit.sample.clientserver.SimpleServerTest (because only received one client call instead of 30). This also means that each of the three SimpleClientTest's testClientCall() method executed successfully.
It is also possible to just use the jrunit framework for running the client side only in the case where a server is already running. The way to accomplish this, as shown in org.jboss.jrunit.sample.clientonly.SampleClientOnlyTest is to change the last parameter to the addTestClasses() method to be null.
public class SampleClientOnlyTest extends TestDriver
{
public void declareTestClasses()
{
addTestClasses("org.jboss.jrunit.sample.basic.SimpleClientTest",
3,
null);
}
}To test this, run the org.jboss.jrunit.sample.basic.SimpleServerTest, then the org.jboss.jrunit.sample.clientonly.SampleClientOnlyTest. Remember that the SimpleServerTest will have to be shutdown manually in this case. The results should be the same as the previous example, except without the failure from the SimpleServerTest.
This sample is exactly like the previous client/server examples from before, except the client is org.jboss.jrunit.sample.decoratedclientserver.SimpleDecoratedClientTest. This class has the same code as the SimpleClientTest in the previous examples, but has added code for obtaining benchmark results in addition to the basic JUnit test results.
... public static Test suite() { return new ThreadLocalDecorator(SimpleDecoratedClientTest.class, 10); } public void testClientCall() throws Exception { // start benchmark tracking ThreadLocalBenchmark.openBench("ClientCall"); ThreadLocalBenchmark.openBench("GetSocket"); getSocket(); ThreadLocalBenchmark.closeBench("GetSocket"); /** * Uncomment this to see that is the same object, thread, and socket for each loop, * but will be different for each thread as specified to the test decorator. This * is based on the values passed to JunitThreadDecorator (or one of it's subclasses) * for the numberOfThreads and loop parameter values. */ //System.out.println(hashCode() + " - " + Thread.currentThread().getName() + " - " + socket.hashCode()); oos.writeObject("This is the request from " + Thread.currentThread().getName()); oos.reset(); oos.writeObject(Boolean.TRUE); oos.flush(); oos.reset(); Object obj = objInputStream.readObject(); objInputStream.readObject(); assertEquals("This is response.", obj); ThreadLocalBenchmark.closeBench("ClientCall"); }
The main differences in code is a new suite() method has been added which creates a new ThreadLocalDecorator, specifing that client should loop running it's test 10 times. There are also lines within the testClientCall() which specify when particular benchmark metrics, via static calls to ThreadLocalBenchmark, should be started and stopped. For more details on ThreadLocalBenchmark, please refer to the section on ThreadLocalBenchmark. The SampleDecoratedClientServerTest class extends BenchmarkTestDriver instead of TestDriver. This sets up the root test driver so that it will listen and report the remote benchmark results to the console and to a local file based on the test's class name (i.e. SampleDecoratedClientServerTest_benchmark.txt).