This weekend I released version 0.9.3 of ScalaTest, a testing tool for Java and Scala programmers. One of the issues that has arisen as I've been working on ScalaTest is what to do about setup and tearDown. These methods, which appeared on class TestCase of the original JUnit, run before and after each test. (JUnit 4 and TestNG use "before" and "after" annotations to serve the same purpose.) This allows you to reinitialize shared mutable objects referenced from instance variables, or other mutable "fixtures" such as files, databases, or sockets, before each test. If one test destroys a fixture, the next test will get a new fixture.

In ScalaTest, I pulled this technique out into a separate trait, which I called ImpSuite. The "Imp" in ImpSuite is short for "imperative," because this technique is definitely an imperative approach to solving the problem. If you want to take this approach in ScalaTest, it is as easy as mixing in trait ImpSuite and overriding methods beforeEach and/or afterEach. However, I wanted to recommend that users consider a few other approaches that avoid reassigning variables (vars) first, because the tradition in Scala programming is to minimize the use of vars.

I wanted to get feedback on these suggestions. I'm including the relevant section of the ScalaTest's Scaladoc next. Please take a look and submit feedback in the forum discussion for this post.

Test fixtures (from ScalaTest's documentation)

A test fixture is objects or other artifacts (such as files, sockets, database connections, etc.) used by tests to do their work. If a fixture is used by only one test method, then the definitions of the fixture objects should be local to the method, such as the objects assigned to sum and diff in the previous MySuite examples. If multiple methods need to share a fixture, the best approach is to assign them to instance variables. Here's a (very contrived) example, in which the object assigned to shared is used by multiple test methods:

 import org.scalatest.Suite

 class MySuite extends Suite {

   // Sharing fixture objects via instance variables
   private val shared = 5

   def testAddition() {
     val sum = 2 + 3
     assert(sum === shared)
   }

   def testSubtraction() {
     val diff = 7 - 2
     assert(diff === shared)
   }
 }
 

In some cases, however, shared mutable fixture objects may be changed by test methods such that it needs to be recreated or reinitialized before each test. Shared resources such as files or database connections may also need to be cleaned up after each test. JUnit offers methods setup and tearDown for this purpose. In ScalaTest, you can use ImpSuite, which will be described later, to implement an approach similar to JUnit's setup and tearDown, however, this approach often involves reassigning vars between tests. Before going that route, you should consider two approaches that avoid vars. One approach is to write one or more "create" methods that return a new instance of a needed object (or a tuple of new instances of multiple objects) each time it is called. You can then call a create method at the beginning of each test method that needs the fixture, storing the fixture object or objects in local variables. Here's an example:

 import org.scalatest.Suite
 import scala.collection.mutable.ListBuffer

 class MySuite extends Suite {

   // create objects needed by tests and return as a tuple
   private def createFixture = (
     new StringBuilder("ScalaTest is "),
     new ListBuffer[String]
   )

   def testEasy() {
     val (builder, lbuf) = createFixture
     builder.append("easy!")
     assert(builder.toString === "ScalaTest is easy!")
     assert(lbuf.isEmpty)
     lbuf += "sweet"
   }

   def testFun() {
     val (builder, lbuf) = createFixture
     builder.append("fun!")
     assert(builder.toString === "ScalaTest is fun!")
     assert(lbuf.isEmpty)
   }
 }
 

Another approach to mutable fixture objects that avoids vars is to create "with" methods, which take test code as a function that takes the fixture objects as parameters, and wrap test code in calls to the "with" method. Here's an example:

 import org.scalatest.Suite
 import scala.collection.mutable.ListBuffer

 class MySuite extends Suite {

   private def withFixture(testFunction: (StringBuilder, ListBuffer[String]) => Unit) {

     // Create needed mutable objects
     val sb = new StringBuilder("ScalaTest is ")
     val lb = new ListBuffer[String]

     // Invoke the test function, passing in the mutable objects
     testFunction(sb, lb)
   }

   def testEasy() {
     withFixture {
       (builder, lbuf) => {
         builder.append("easy!")
         assert(builder.toString === "ScalaTest is easy!")
         assert(lbuf.isEmpty)
         lbuf += "sweet"
       }
     }
   }

   def testFun() {
     withFixture {
       (builder, lbuf) => {
         builder.append("fun!")
         assert(builder.toString === "ScalaTest is fun!")
         assert(lbuf.isEmpty)
       }
     }
   }
 }

 import org.scalatest.Suite
 import java.io.FileReader
 import java.io.FileWriter
 import java.io.File

 class MySuite extends Suite {

   private def withTempFile(testFunction: FileReader => Unit) {

     val FileName = "TempFile.txt"

     // Set up the temp file needed by the test
     val writer = new FileWriter(FileName)
     try {
       writer.write("Hello, test!")
     }
     finally {
       writer.close()
     }

     // Create the reader needed by the test
     val reader = new FileReader(FileName)

     try {
       // Run the test using the temp file
       testFunction(reader)
     }
     finally {
       // Close and delete the temp file
       reader.close()
       val file = new File(FileName)
       file.delete()
     }
   }

   def testReadingFromTheTempFile() {
     withTempFile {
       (reader) => {
         var builder = new StringBuilder
         var c = reader.read()
         while (c != -1) {
           builder.append(c.toChar)
           c = reader.read()
         }
         assert(builder.toString === "Hello, test!")
       }
     }
   }

   def testFirstCharOfTheTempFile() {
     withTempFile {
       (reader) => {
         assert(reader.read() === 'H')
       }
     }
   }
 }
 

If you are more comfortable with reassigning instance variables, however, you can instead use ImpSuite, a subtrait of Suite that provides methods that will be run before and after each test. ImpSuite's beforeEach method will be run before, and its afterEach method after, each test (like JUnit's setup and tearDown methods, respectively). For example, here's how you'd write the previous test that uses a temp file with an ImpSuite:

 import org.scalatest.ImpSuite
 import java.io.FileReader
 import java.io.FileWriter
 import java.io.File

 class MySuite extends ImpSuite {

   private val FileName = "TempFile.txt"
   private var reader: FileReader = _

   // Set up the temp file needed by the test
   override def beforeEach() {
     val writer = new FileWriter(FileName)
     try {
       writer.write("Hello, test!")
     }
     finally {
       writer.close()
     }

     // Create the reader needed by the test
     reader = new FileReader(FileName)
   }

   // Close and delete the temp file
   override def afterEach() {
     reader.close()
     val file = new File(FileName)
     file.delete()
   }

   def testReadingFromTheTempFile() {
     var builder = new StringBuilder
     var c = reader.read()
     while (c != -1) {
       builder.append(c.toChar)
       c = reader.read()
     }
     assert(builder.toString === "Hello, test!")
   }

   def testFirstCharOfTheTempFile() {
     assert(reader.read() === 'H')
   }
 }
 

In this example, the instance variable reader is a var, so it can be reinitialized between tests by the beforeEach method. If you want to execute code before and after all tests (and nested suites) in a suite, such as you could do with @BeforeClass and @AfterClass annotations in JUnit 4, you can use the beforeAll and afterAll methods of ImpSuite.

See the documentation for ScalaTest for more information.

What's your opinion?

Would you want to write tests this way? Would you stick with before and after? Do you see any benefit in trying to avoid vars like this?

import org.scalatest.fun.FunSuite1

 class EasySuite extends FunSuite1[StringBuilder] {

   testWithFixture("easy test") {
     sb => {
       sb.append("easy!")
       assert(sb.toString === "Testing is easy!")
     }
   }

   testWithFixture("fun test") {
     sb => {
       sb.append("fun!")
       assert(sb.toString === "Testing is fun!")
     }
   }

   def withFixture(f: StringBuilder => Unit) {
     val sb = new StringBuilder("Testing is ")
     f(sb)
   }
 }

import org.scalatest.fun.FunSuite1

 class EasySuite extends FunSuite1[StringBuilder] {

   test("easy test") {
     sb => {
       sb.append("easy!")
       assert(sb.toString === "Testing is easy!")
     }
   }

   test("fun test") {
     sb => {
       sb.append("fun!")
       assert(sb.toString === "Testing is fun!")
     }
   }

   def withFixture(f: StringBuilder => Unit) {
     val sb = new StringBuilder("Testing is ")
     f(sb)
   }
 }

class MySuite extends Suite {
  var theTestThisCalled = false
  var theTestThatCalled = false
  def testThis() { theTestThisCalled = true }
  def testThat() { theTestThatCalled = true }
}