|
|
|
Sponsored Link •
|
Summary
Today I released a new ScalaTest-1.5 snapshot release that contains several enhancements, including formatted output for TDD-style traits, indented output for nested style traits, and two new style traits, PropSpec and FreeSpec.
|
Advertisement
|
Today I released an updated ScalaTest-1.5-SNAPSHOT that includes several enhancements, including two new style traits, PropSpec and FreeSpec. The snapshot works with Scala 2.8. You can download the snapshot release via the scala-tools.org Maven repository with:
Or you can just grab the jar file from:
I put the Scaladoc for this snapshot release up here:
http://www.artima.com/docs-scalatest-1.5-SNAPSHOT-25-Apr-2011/
PropSpec trait
PropSpec allows you to create a suite of property-based tests. It works the same as a FunSuite, but instead of test you write property, and instead of testsFor you write propertiesFor.
PropSpec supports both ScalaCheck and ScalaTest property styles (as well as any other property style you may wish to use it with.) If you want to write properties in the ScalaCheck style, mix Checkers into your PropSpec. If you want to write them in the ScalaTest style, mix in PropertyChecks. Here's an example that uses both generator- and table-driven property checks:
import org.scalatest.PropSpec import org.scalatest.prop.PropertyChecks import org.scalatest.matchers.ShouldMatchers class FractionSpec extends PropSpec with PropertyChecks with ShouldMatchers {
property("Fraction constructor normalizes numerator and denominator") {
forAll { (n: Int, d: Int) => whenever (d != 0 && d != Integer.MIN_VALUE && n != Integer.MIN_VALUE) {
val f = new Fraction(n, d)
if (n < 0 && d < 0 || n > 0 && d > 0) f.numer should be > 0 else if (n != 0) f.numer should be < 0 else f.numer should be === 0
f.denom should be > 0 } } }
property("Fraction constructor throws IAE on bad data.") {
val invalidCombos = Table( ("n", "d"), (Integer.MIN_VALUE, Integer.MIN_VALUE), (1, Integer.MIN_VALUE), (Integer.MIN_VALUE, 1), (Integer.MIN_VALUE, 0), (1, 0) )
forAll (invalidCombos) { (n: Int, d: Int) => evaluating { new Fraction(n, d) } should produce [IllegalArgumentException] } } }
For more information, see the Scaladoc documentation for PropSpec and my earlier post, ScalaTest Property Checks Preview. Trait PropSpec is the last piece of ScalaTest's new support for property-based testing described in that previous post.
Note: Trait PropSpec is in part inspired by class org.scalacheck.Properties, designed by Rickard Nilsson for the ScalaCheck test framework.
FreeSpec trait
The other new style trait introduced in today's snapshot release is FreeSpec. Whereas ScalaTest's other specification-style traits facilitate writing text with certain grammatical structures using words like "when," "should," and "can," FreeSpec allows you to structure the text of your specification however you wish. You write a test in a FreeSpec with a string followed by in and a block of test code, just as you do in WordSpec and FlatSpec:
"should pop values in last-in-first-out order" in { // ... }
You can surround tests with description clauses composed of a string, a dash character (-), and a block. Here's an example:
"A Stack" - { "should pop values in last-in-first-out order" in { // ... } }
You can nest description clauses inside description clauses to any number of levels. Here's an example:
import org.scalatest.FreeSpec class StackSpec extends FreeSpec { "A Stack" - { "whenever it is empty" - { "certainly ought to" - { "be empty" in { // ... } "complain on peek" in { // ... } "complain on pop" in { // ... } } } "but when full, by contrast, must" - { "be full" in { // ... } "complain on push" in { // ... } } } }
When run in the interpreter, you'd see:
scala> (new StackSpec).execute() StackSpec: A Stack whenever it is empty certainly ought to - be empty - complain on peek - complain on pop but when full, by contrast, must - be full - complain on push
Another use case for FreeSpec is writing specification-style test suites in a language other than English, as demonstrated here:
import org.scalatest.FreeSpecRunning the above
class ComputerRoomRulesSpec extends FreeSpec { "Achtung!" - { "Alle touristen und non-technischen lookenpeepers!" - { "Das machine is nicht fuer fingerpoken und mittengrabben." in { // ... } "Is easy" - { "schnappen der springenwerk" in { // ... } "blowenfusen" in { // ... } "und poppencorken mit spitzen sparken." in { // ... } } "Das machine is diggen by experten only." in { // ... } "Is nicht fuer gerwerken by das dummkopfen." in { // ... } "Das rubbernecken sightseeren keepen das cottenpicken hands in das pockets." in { // ... } "Relaxen und watchen das blinkenlights." in { // ... } } } }
ComputerRoomRulesSpec in the Scala interpreter would give you:
scala> (new ComputerRoomRulesSpec).execute() ComputerRoomRulesSpec: Achtung! Alle touristen und non-technischen lookenpeepers! - Das machine is nicht fuer fingerpoken und mittengrabben. Is easy - schnappen der springenwerk - blowenfusen - und poppencorken mit spitzen sparken. - Das machine is diggen by experten only. - Is nicht fuer gerwerken by das dummkopfen. - Das rubbernecken sightseeren keepen das cottenpicken hands in das pockets. - Relaxen und watchen das blinkenlights.
The FreeSpec concept first saw light of day way back in ScalaTest 0.9.4 under the name SpecDasher. I released it already deprecated with a warning that I would remove it in 0.9.5, which I did. I only released it because I'd shown it in Programming in Scala, which had already gone to the printer. I wanted all the code in the book to work for the version of ScalaTest mentioned in the book. I removed it in 0.9.5 because I wasn't convinced I had figured out the best way to do it, and even if I figured that out, I only planned to add it if users actually convinced me it would be useful to them.
Eric Torreborre's specs framework always had a >> operator that provided a similar nesting ability to FreeSpec's dash character, but in the specs case >> was an alias for in, which meant tests (called "examples" in specs) were being nested not specification text (and you still could put a should on top). The specsy project by Esko Luontola came closer to the concept, and in fact Esko wrote a blog post about the troubles with pre-defined words, Choice of Words in Testing Frameworks. Over time I did get input from users that they would find value in this kind of trait, for example, in this scalatest-users discussion started by Sukant Hajra. And I got the opposite feedback, in a way, of seeing someone use trait Spec and completely ignoring the guiding structure. So now FreeSpec is here.
Another user feedback that I got over time is that users really liked the formatted output of the BDD-style (Behavior-Driven-Development-style) traits, and wanted to see the same thing in the TDD-style (Test-Driven-Development-style) traits. I did things that way originally because making the output of running a test suite a more useful artifact was a push from the BDD folks. But the users gave me this feedback, so now even when you run a Suite, FunSuite, JUnitSuite, JUnit3Suite, or a TestNGSuite, you get nicely formatted output. For example, given this FunSuite:
import org.scalatest.FunSuite
class MySuite extends FunSuite {
test("addition") { val sum = 1 + 1 assert(sum === 2) assert(sum + 2 === 4) }
test("subtraction") { val diff = 4 - 1 assert(diff === 3) assert(diff - 2 === 1) } }
Running from the interpreter in ScalaTest 1.3 gives you:
scala> (new MySuite).execute() Test Starting - MySuite: addition Test Succeeded - MySuite: addition Test Starting - MySuite: subtraction Test Succeeded - MySuite: subtraction
But as of the latest ScalaTest 1.5 snapshot release will give you:
scala> (new MySuite).execute() MySuite: - addition - subtraction
Another bit of user feedback was that people preferred to see the nested levels in the test class echoed in indentation levels in the output. ScalaTest has supported arbitrary levels of indentation since 1.0 in its event hierarchy, but didn't fire indentation of more than one level from any ScalaTest trait, not even from Spec, which allowed arbitrarily deep levels of nesting. My thought was that most often people wouldn't actually nest more deeply than two levels, and I felt that it was more readable to flatten two levels to one. Also I observated Ruby's RSpec tool, which inspired Spec's describe/it syntax, at the time flattened everything to one level. I wasn't sure which way to go, so I followed RSpec initially. Meanwhile Eric Torreborre indented specs output to match the input, and I observed that led some specs users to nest text more deeply, and even RSpec now indents its output this way. The ScalaTest users I asked about this for the most part indicated they would like the indented output. So now given this Spec:
import org.scalatest.Spec
class MySpec extends Spec { describe("A Stack") { describe("(when empty)") { it("should be empty") (pending) it("should complain on peek") (pending) it("should complain on pop") (pending) } describe("(when full)") { it("should be full") (pending) it("should complain on a push") (pending) } } }
Instead of getting the output you get with ScalaTest 1.3:
scala> (new MySpec).execute() A Stack (when empty) - should be empty (pending) - should complain on peek (pending) - should complain on pop (pending) A Stack (when full) - should be full (pending) - should complain on a push (pending)
As of the latest ScalaTest 1.5 snapshot, you will get:
scala> (new MySpec).execute() MySpec: A Stack (when empty) - should be empty (pending) - should complain on peek (pending) - should complain on pop (pending) (when full) - should be full (pending) - should complain on a push (pending)
For two levels of indentation, it isn't necessarily an improvement in readability, but at more levels it is more clearly a readability win. For example, given this WordSpec:
import org.scalatest.WordSpec class ScalaTestGUISpec extends WordSpec {
def theUser = afterWord("the user") def display = afterWord("display") def is = afterWord("is")
"The ScalaTest GUI" when theUser { "clicks on an event report in the list box" should display { "a blue background in the clicked-on row in the list box" in {} "the details for the event in the details area" in {} "a rerun button" that is { "enabled if the clicked-on event is rerunnable" in {} "disabled if the clicked-on event is not rerunnable" in {} } } } }
Instead of the ScalaTest 1.3 output of:
scala> (new ScalaTestGUISpec).execute() The ScalaTest GUI (when the user clicks on an event report in the list box) - should display a blue background in the clicked-on row in the list box - should display the details for the event in the details area - should display a rerun button that is enabled if the clicked-on event is rerunnable - should display a rerun button that is disabled if the clicked-on event is not rerunnable
As of the latest ScalaTest 1.5 snapshot, you'll get:
scala> (new ScalaTestGUISpec).execute() ScalaTestGUISpec: The ScalaTest GUI when the user clicks on an event report in the list box should display - a blue background in the clicked-on row in the list box - the details for the event in the details area a rerun button that is - enabled if the clicked-on event is rerunnable - disabled if the clicked-on event is not rerunnable
These enhancements will be released as part of ScalaTest 1.5 within the next few weeks. I'm posting this preview now because I want to get feedback in general on the API and find if there are any bugs to fix or any code-breakages. (I expect no source code to break with any of these enhancements, so let me know if you have a problem.) So please give it a try and either post feedback to the discussion for for this blog post, or email the scalatest-users mailing list.
|
Advertisement
|
||||||
Have an opinion? Be the first to post a comment about this weblog entry.
If you'd like to be notified whenever Bill Venners adds a new entry to his weblog, subscribe to his RSS feed.
 | Bill Venners is president of Artima, Inc., publisher of Artima Developer (www.artima.com). He is author of the book, Inside the Java Virtual Machine, a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Active in the Jini Community since its inception, Bill led the Jini Community's ServiceUI project, whose ServiceUI API became the de facto standard way to associate user interfaces to Jini services. Bill is also the lead developer and designer of ScalaTest, an open source testing tool for Scala and Java developers, and coauthor with Martin Odersky and Lex Spoon of the book, Programming in Scala. |
|
Sponsored Links
|
Digg
del.icio.us
Reddit