JDistUnit

In this section, we get to the nuts and bolts of using JDistUnit. I assume that you have the jdistunit.jar and the agentopia.jar files in your classpath and the test servers happily humming along.

Programming the Test Action

Within your JUnit test class, you will probably have a testSomething() (JUnit3) or a @Test annotated (JUnit4) method. To use JDistUnit, you will need to move that method into your own subclass of JDistUnitAgent, into a method called testAction().

@AgentopiaAgentMarker(name = "My Test Agent")
public class MyTestAgent extends JDistUnitAgent {

    public void testAction(ITestThread testThread) throws Throwable {
        // My test code.
    }

}

At the top, you see the @AgentopiaAgentMarker, which tells JDistUnit that this is a class whose byte code should be loaded over the network. The agent class simply extends JDistUnitAgent, and the only thing to be added is the test code within the testAction() method.

In that method header, you see that it may throw any Exception or Error (which have the common Throwable superclass). This is necessary because any fault can happen on the test servers during the test action, and you still want to get your agent back home to explain that there was an error. The ITestThread parameter is explained later.

The UML sequence diagram below clarifies how the agent, the test thread and the test agent work together to run your testAction() after arriving on each of the test servers.

At the top, the diagram shows agent and test action as separate concepts although they are actually one; this is in order to highlight agent vs. test action activities: First, the agent starts e.g. 1000 test threads, giving itself (i.e. the test action) as parameter. Then, each ITestThread starts the action once, giving itself to your test method.

Thus, the testAction() gets the ITestThread as a parameter. This is so because the test thread may time out; any results you deliver thereafter will be ignored. To avoid unnecessary runtime consumption on the test servers, I recommend you check ITestThread.isActionStopped() regularly, and just close all connections and return if this is the case.

And now we return to the JUnit test case and actually use your agent + action.

Programming the Test Case with Agents

Back in the test case, the first thing I recommend is to enhance the setUp() and tearDown() methods to create and remove the ITestBox which will run your test agent on the test servers.

public class JDistUnitExampleTest extends TestCase {

    private ITestBox testBox;

    protected void setUp() throws Exception {
        testBox = new TestBox(new HostId("ginko.vpn:15900"));
        testBox.connectToNetwork(new HostId("ginko.vpn:15907"));
        testBox.connectToNetwork(new HostId("eva.vpn:15907"));
    }

    protected void tearDown() throws Exception {
        testBox.shutdown();
        Thread.sleep(2000);
    }

}

In the JUnit3 TestCase above, I create a testbox with a local hostname that returning agents can connect to (so please do not use "localhost:8080" there). Then, I connect the test box to two test servers on my VPN: One is the same computer, but on another port; the other is an entirely different machine.

Complementary to what happens at setUp(), I need to shut down the test box at tearDown(). There is a small complication: The operating system underneath needs some time to release the network port resource. The required time is dependent on the OS, so to be on the safe side, use e.g. 2 seconds.

Then, in the actual test method, create the agent and use the ITestBox to deploy it.

    public void testHomepageLoad() throws Exception {
        JDistUnitAgent agent = new HomepageTestAgent();
        agent.setRequestCount(2);
        agent.setTimeOut(30000L);
        agent.setThreadTimeStep(300L);

        testBox.deployTestAgents(agent);
        testBox.waitForResults(1);

        assertEquals(testBox.getTestServerCount(), testBox.getReturnedAgentCount());
        ...
    }

In the example above, I create a HomePageTestAgent, which just reads the JDistUnit homepage via URL stream. It shall perform 2 requests on each test server, and all combined page loads shall not take longer than 30 seconds. Since one dedicated ITestThread is started for each request, I keep those thread starts 300 ms apart, such that the web server does not suspect a DOS attack.

Using the ITestBox, I deploy the test agent (as multiple copies) to the previously connected test servers. Then, I wait at maximum 1 minute for the agents to return. If all agents returned, or the minute passed, the test continues. Find out whether all agents returned by comparing the number of test servers with the number of returned agents.

And then, the measurements can commence. We start with the "stress testing" part (i.e. how many requests were successful, how many failed, how many timed out):

    public void testHomepageLoad() throws Exception {
        ...
        assertEquals(4, testBox.getNumberOfSuccesses());
        assertEquals(0, testBox.getNumberOfFailures());
        assertEquals(0, testBox.getNumberOfStarvations());

        List<Throwable> errorList = testBox.getErrorList();
        assertTrue(errorList.isEmpty());
        ...
    }

The test box is asked for number of successes (finished gracefully), failures (threw exceptions) and starvations (30 second timeout occurred). We expect all 4 requests (2 requests, 2 test servers) to be successful.

If failures occurred, the test box can supply all Throwables in a list. This is possible because all exceptions and errors in Java are Serializable and thus can be transmitted over the network.

And finally, the "load testing" part (i.e. how fast was everything). For the purposes of performance measusurements, failed or timed-out requests are ignored.

    public void testHomepageLoad() throws Exception {
        ...
        assertTrue(testBox.getPerformanceMinimum() > 50);
        assertTrue(testBox.getPerformanceAverage() < 3000);
        assertTrue(testBox.getPerformanceMaximum() < 5000);
        ...
    }

For all successful requests, the test box calculates minimum, maximum, and average times in milliseconds. Above, I require that all access times are over 50 ms, no request takes longer than 5 s, and that on average requests take no longer than 3s.

In summary, the UML sequence diagram below shows what happens during the test execution over the network.

In the TestCase, you prepare by creating a test box, and connecting it to all available test servers. You then create your test agent (which also embeds your ITestAction), and deploy it to the test box. The test box will now copy the agent to all test servers, where each agent will run the test action.

Back on your machine, you wait for the agents to return, and after they do, you ask for number of sucesses / failures / starvations (stress testing), and for minimum / maximum / average access times (load testing).

And that would be it! If you have any further questions or comments, you can go to the Contact section and write me an email.