---

Artima SuiteRunner is a free open source testing toolkit for Java released under the Open Software License. You can use this tool with JUnit to run existing JUnit test suites, or standalone to create unit and conformance tests for Java APIs. To get you up and running quickly, the Artima SuiteRunner distribution ZIP file includes a simple example. This article uses the example to show you how to use Artima SuiteRunner to run existing JUnit test suites.

When you unzip the Artima SuiteRunner distribution ZIP file, you get the following items:

Name	Contents
account.srj	Recipe file for the example
accountjunit.srj	Recipe file for the JUnit example (Added in 1.0beta4)
example	Directory of Java source and class files for the example
javadoc	Directory containing Javadoc documentation for the API
README.txt	The obligatory README file
suiterunner-[release].jar	Artima SuiteRunner executable JAR file
suiterunner-[release]-src.zip	ZIP file of Artima SuiteRunner source code
xmlreporter.srj	Recipe file for the XMLReporter example (Added in 1.0beta5)

Artima SuiteRunner's executable JAR file is suiterunner-[release].jar. The source and class files for the example are found in the example directory. A recipe file that runs the example JUnit test suite is contained in accountjunit.srj, which was added to the distribution in 1.0beta4. If you don't see accountjunit.srj, then you need to download the latest version of Artima SuiteRunner.

Starting the Graphical User Interface

This article shows screenshots of Artima SuiteRunner version 1.0beta4, as it appeared on Mac OS X. To run the example JUnit test suite, type this command in the suiterunner-[release] subdirectory of the directory in which you unzipped the Artima SuiteRunner distribution ZIP file:

java -jar suiterunner-[release].jar accountjunit.srj

When you type the previous command, you should see:

Figure 1. Without junit.jar in the runpath, the run aborts.

When you first start Artima SuiteRunner with the previous command, it attempts to run the TestCase mentioned in the recipe file. As shown in Figure 1, this initial run attempt aborts. To see the information shown in Figure 1 in the Details text area, click on "Run Aborted" in the Reports list. The details indicate that Artima SuiteRunner aborted your requested run because it couldn't find class junit.framework.TestCase.

Understanding the Recipe File

In the following command, the first and only argument to the Artima SuiteRunner executable JAR file is the name of a recipe file, accountjunit.srj:

java -jar suiterunner-[release].jar accountjunit.srj

A recipe file is a Java properties file that supplies information needed by Artima SuiteRunner to run a suite of tests. Recipe files contain values for three properties: org.suiterunner.Runpath, org.suiterunner.Suites, and org.suiterunner.Reporters. The contents of accountjunit.srj are:

org.suiterunner.Runpath=-p "example"
org.suiterunner.Suites=-s com.artima.examples.account.ex6test.AccountTestCase
org.suiterunner.Reporters=-g

In accountjunit.srj:

• The value of org.suiterunner.Runpath (-p "example") specifies a runpath with a single directory, example.
• The value of org.suiterunner.Suites (-s com.artima.examples.account.ex6test.AccountTestCase) indicates that Artima SuiteRunner should load the specified class, a subclass of junit.framework.TestCase, and run it.
• The value of org.suiterunner.Reporters (-g) indicates that Artima SuiteRunner will display its graphical user interface (GUI) and show the results of the run there.

When invoked via the previous command that specifies accountjunit.srj as a command line parameter, Artima SuiteRunner will:

1. Create a URLClassLoader that can load classes from the example directory, the directory specified in the recipe file's org.suiterunner.Runpath property.
2. Via the URLClassLoader, attempt to load class com.artima.examples.account.ex6test.AccountTestCase, the class specified in in the recipe file's org.suiterunner.Suites property.
3. Discover via a NoClassDefFoundError that the com.artima.examples.account.ex6test.AccountTestCase class can't be loaded because its superclass, junit.framework.TestCase, can't be found.
4. Abort the run and reports the NoClassDefFoundError to the user.

Adding junit.jar to the Runpath

When you type the following command, Artima SuiteRunner aborts because it can't load class junit.framework.TestCase:

java -jar suiterunner-[release].jar accountjunit.srj

To enable Artima SuiteRunner to load junit.framework.TestCase and the rest of the JUnit classes, you must tell Artima SuiteRunner where to find junit.jar. JUnit is not included in the Artima SuiteRunner distribution ZIP file. Presumably, if you want to use Artima SuiteRunner as a JUnit runner, you have already JUnit installed on your system. If you don't, you can download JUnit from junit.org. Once you locate junit.jar on your system, you can add it to Artima SuiteRunner's runpath by editing the recipe. To edit the recipe, select File-->Edit Recipe, as shown in Figure 2.

Figure 2. Editing the recipe.

The Edit Recipe dialog allows you to edit all three components of the recipe: suites, runpath, and reporters. To add junit.jar to the runpath, you must select the Runpath tab, as shown in Figure 3.

Figure 3. Selecting the Runpath tab.

The easiest way to add junit.jar to the runpath is by pressing on the Select button on the right hand side of the Runpath tab's dialog. Pressing Select brings up a file chooser dialog with the title "Select JAR File or Directory to Add." Use this dialog box to navigate to your copy of junit.jar, as shown in Figure 4, then press Choose.

Figure 4. Selecting junit.jar to add to the runpath.

On my Macintosh, junit.jar is located in the /Users/bv/nobkp/junit directory. As Figure 5 shows, junit.jar is now in the runpath. When you press OK, Artima SuiteRunner will change the runpath per your request, and attempt to run AccountTestCase again.

Figure 5. File junit.jar is now in the runpath.

When you press the OK button of the Edit Recipe dialog, the AccountTestCase should run and give the results shown in Figure 6. Don't be disheartened by all the red you see in the Reports list. The example is supposed to fail initially. It is left as an exercise for you to fix the bugs in the account example to get the test to run successfully. The existence of the Run Starting, Test Failed, and Run Completed reports in the Reports list indicates that Artima SuiteRunner successfully loaded the JUnit classes and executed the example JUnit test suite.

Figure 6. With junit.jar in the runpath, the AccountTestCase example runs.

Changing the Runpath to Run Your Existing JUnit Test Suite

In the previous step, you used Artima SuiteRunner as a JUnit runner to run the example JUnit test suite included in the Artima SuiteRunner distribution. Now I'll show you how to use Artima SuiteRunner to run your own existing JUnit test suite. To do so, you must edit the recipe and change the runpath and suites list.

To edit the runpath, select File-->Edit Recipe and click on the Runpath tab. The Runpath tab's dialog should still show the example directory and the junit.jar file. To run your existing JUnit test suite, you must keep junit.jar in the runpath, but you can get rid of the example directory. To remove the example directory from the runpath, click on example and press the Remove button, as shown in Figure 7.

Figure 7. Removing example from the runpath.

After pressing the Remove button, only junit.jar remains in the runpath, as shown in Figure 8.

Figure 8. The example directory is no longer in the runpath.

At this point, you must add to the runpath all JAR files and directories needed by JUnit to run your existing test suite. If you use JUnit's Swing runner, for example, you will likely need to add to the runpath all the files and directories that appear in the classpath when you run JUnit. To load classes for your existing JUnit test suite, Artima SuiteRunner will create a URLClassLoader that will look in each JAR file and directory in the order they appear in the runpath.

In my case, I have a JUnit-based conformance test suite for the ServiceUI API sitting in the directory /Users/bv/projects/scslsrc/serviceui. (This is the very JUnit test suite that gave me sufficient frustration that I went off and started writing what eventually became Artima SuiteRunner, as described in Why We Refactored JUnit.) Because both the classes that make up the ServiceUI API conformance test suite and the ServiceUI API classes being tested can be loaded from the /Users/bv/projects/scslsrc/serviceui directory, all I need to do is to add the /Users/bv/projects/scslsrc/serviceui directory to the runpath. To add this directory, I press the Select button on the Runpath tab's dialog, navigate to the /Users/bv/projects/scslsrc directory, select serviceui, and press the Choose button, as shown in Figure 9.

Figure 9. Selecting the serviceui directory to add to the runpath.

As shown in Figure 10, my runpath now includes junit.jar and the directory from which Artima SuiteRunner can load both the class files for the JUnit-based ServiceUI API conformance test suite and the actual ServiceUI API classes they test.

Figure 10. The serviceui directory is now in the runpath.

Selecting Your Existing JUnit TestCase

At this point, my runpath will enable all the classes needed by the JUnit-based ServiceUI API conformance test suite to be loaded, but my recipe is still not quite ready. The recipe's suites list still indicates Artima SuiteRunner should run class com.artima.examples.ex6.ex6test.AccountTestCase, the class from the example JUnit test suite specified in accountjunit.srj. Hopefully, you have built the correct runpath for your own existing JUnit test suite. Your only remaining step is to specify the appropriate suites list.

As a first step, you can remove the com.artima.examples.ex6.ex6test.AccountTestCase class from the suites list. If you press the Suites tab in the Edit Recipe dialog, you will see the current suites list, as shown in Figure 11.

Figure 11. Removing AccountTestCase from the suites list.

Click on AccountTestCase in the suites list and press the Remove button. This should leave no suites specified, as shown in Figure 12. Pressing the Edit Recipe's OK button will cause Artima SuiteRunner to change the recipe as you specified, with a new runpath and empty suites list.

Figure 12. The suites list is now empty.

At this point, the easiest way to select a JUnit TestCase to run is by pressing the Select button at the top of Artima SuiteRunner's main window. As shown in Figure 13, Artima SuiteRunner will search the runpath for any org.suiterunner.Suite classes or junit.framework.TestCase classes and display them in the "Select a Suite to Run" dialog's Suites list box.

If you don't see the TestCase you want to run in this list, that means you have an error in your runpath. Make sure the JAR file or directory that contains the TestCase class you want to run appears in the runpath.

In my case, Artima SuiteRunner discovers all the TestCases in the JUnit-based ServiceUI API conformance test suite, because they are all available in the directory I added to the runpath, /Users/bv/projects/scslsrc/serviceui. To indicate to Artima SuiteRunner that I want to run LocalesTestCase, I select LocalesTestCase in the Suites list and press OK, as shown in Figure 13.

Figure 13. Selecting LocalesTestCase via the Suite Selector dialog.

As soon as I press the Select a Suite to Run dialog's OK button, Artima SuiteRunner attempts to load the class I selected, LocalesTestCase, and run it. As shown in Figure 14, Artima SuiteRunner is able to load and run this JUnit TestCase, and what's more, the test itself succeeds with a big satisfying green bar.

Figure 14. The LocalesTestCase runs with no errors.

Creating a Recipe File for Your Existing JUnit Test Suite

Once you get the recipe correct, Artima SuiteRunner will run your existing JUnit test suite. To make it easier to run this test suite again in the future, you should save the current recipe in a file. The current recipe file name, which appears in the title bar of Artima SuiteRunner's main window, is accountjunit.srj. This recipe file, which you specified when you started Artima SuiteRunner, contains the recipe for running the account JUnit test suite example. If you select File-->Save, you will overwrite accountjunit.srj with your new recipe. Most likely, you'll want to create a new recipe file by selecting File-->Save As, as shown in Figure 15.

Figure 15. Saving the current recipe in a file.

You can name the recipe file anything you want, but the default extension for a recipe file is .srj (for SuiteRunner for Java). In my case, I decided to name the file serviceui.srj, as shown in Figure 16.

Figure 16. Naming the recipe file serviceui.srj.

Once you save your new recipe as a file, the new file name will appear in the title bar of Artima SuiteRunner's main window. In my case, the name serviceui.srj appears in Artima SuiteRunner's title bar, as shown in Figure 17.

Figure 17. Running from the serviceui.srj recipe file.

Once you have saved your recipe in a file, you can rerun the test specified by the recipe by starting Artima SuiteRunner with the name of your new recipe file as the first argument. If you associate the extension .srj with the command java -jar suiterunner-[release].jar on your desktop, then you can rerun tests by simply double clicking on the recipe file.

Summary

This article gives an introduction to using Artima SuiteRunner as a JUnit runner to run existing JUnit test suites, but doesn't cover all possible ways to use Artima SuiteRunner in conjunction with JUnit. For example, I could have used the following command to run my existing JUnit-based ServiceUI API conformance test suite completely from the classpath with no runpath specified:

java -classpath ./suiterunner-1.0beta4.jar:/Users/bv/nobkp/junit/junit.jar:/Users/bv/projects/scslsrc/serviceui org.suiterunner.Runner -s com.artima.test.serviceui.ui.attribute.LocalesTestCase

To find out more about how Artima SuiteRunner can help you create and run unit and conformance tests, please see the Artima SuiteRunner Tutorial.

Get Help in the SuiteRunner Forum

For help with Artima SuiteRunner, please post to the SuiteRunner Forum.

Resources

JUnit is available at:
http://www.junit.org

Why We Refactored JUnit
http://www.artima.com/suiterunner/why.html

Artima SuiteRunner Tutorial, Building Conformance and Unit Tests with Artima SuiteRunner:
http://www.artima.com/suiterunner/tutorial.html

Create an XML Reporter for Your Unit Tests, how to create a customer reporter for Artima SuiteRunner that formats unit test results in XML:
http://www.artima.com/suiterunner/xmlreporter.html

Artima SuiteRunner home page:
http://www.artima.com/suiterunner/index.html

Artima SuiteRunner download page (You must log onto Artima.com to download the release):
http://www.artima.com/suiterunner/download.jsp

The SuiteRunner Forum:
http://www.artima.com/forums/forum.jsp?forum=61

Talk back!

Have an opinion? Be the first to post a comment about this article.

About the author

Bill Venners is president of Artima Software, Inc. and editor-in-chief of Artima.com. He is author of the book, Inside the Java Virtual Machine, a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Bill has been active in the Jini Community since its inception. He led the Jini Community's ServiceUI project that produced the ServiceUI API. The ServiceUI became the de facto standard way to associate user interfaces to Jini services, and was the first Jini community standard approved via the Jini Decision Process. Bill also serves as an elected member of the Jini Community's initial Technical Oversight Committee (TOC), and in this role helped to define the governance process for the community. He currently devotes most of his energy to building Artima.com into an ever more useful resource for developers.