This post originated from an RSS feed registered with Agile Buzz by James Robertson. Original Post: SUnitToo Feed Title: Travis Griggs - Blog Feed URL: http://www.cincomsmalltalk.com/rssBlog/travis-rss.xml Feed Description: This TAG Line is Extra Latest Agile Buzz Posts Latest Agile Buzz Posts by James Robertson Latest Posts From Travis Griggs - Blog

!In The Beginning

This journey began, because we were having problems with memory usage under SUnit. The problem is really pretty simple. TestCases are created and store in TestSuites and subsequently in TestResults. A TestCase specifies not only a method to run, but holds all the state created during the run. Unless tearDown always nils out all of its variables, at then end of a run, a TestResult will still hold all of those TestCases. Lets say you're running 1000 tests. And each test allocates and retains objects amounting to an average of 10,000 bytes. Said run will allocate AND retain 10MB. In our cases, we have some tests that work with 4MB Image objects. It doesn't take long for this to get really out of hand.

Steve Aldred and WithStyle/Wizard buddies had posted a fix which used a similiar technique to that used by ExtraIcons. My original goal was to load the latest SourceForge version (Niall Ross is good enough to mirror the source forge project's version as a Open Repository package), and then integrate this change, thus getting rid of the need for a patch package. We've had real issues with the open ended memory consumption and it looked to benefit us.

!Two Birds with One Stone?

At the same time, there was this issue that has bugged me about SUnit for a while. As a tool writer for SUnit, I've found that there's one method I've had to override 2 times: TestResult>>runCase. ExtraIcons does it (it wants to know globally about any tests run, regardless of where from). A logging tool I wrote at Key does something similiar. The memory fix did it (and as a result the other two had to be revisisted AND there's a now a load order dependency). Overrides are great and all, but they don't scale well. When everyone's having to hack one point of your system to plug their behavior in, then there's an opportunity (and apparent necessity) for a more pluggable architecture around that code point (an example of this was with the work done with ExtraEmphases; tools kept hacking the same two methods to get their work done... something more pluggable was needed).

My thought was, as long as I've got this thing open, why not see if I can't fix this problem too?

!Scope Creeps a Little

As I began working on it, I found that I was frustrated working with the SUnitPreload package, which attempts to provide a "compatiblity" layer between different flavors, so that the core can remain highly portable. I was undecided as to how to add to the compatibility layer to add the functionality I needed. I don't have access to all of the other Smalltalks, and in some ways I don't care as much as I used to. Sorry, but the fervor of cross-dialect compatibility just doesn't seem to be there anymore. So... I began inlining things. Man I love the RB. I thought, maybe I'll make a totally compatible version, but that is just VisualWorks specific, at least to begin with.

!The Real Journey Begins

As I continued on, I began to want to play with other things. It's not that SUnit is deficient in any way. I just found myself wanting to try new things. I find myself remembering Alan Kay's "burn the disk pack" adage. As posited a couple of days ago, I wondered what would happen if XP principles were applied to SUnit today, in light of what tools no longer exist, and which do (at least for me).

I struggled with this alot. I went back and forth at least 20 times. One minute, I was going to do a separate compatible version. The next I wasn't. The next, I was going to roll it all into the mainstream one. Or scale back. I was all over the map. One of the frustrating factors was an effort at one point, to "expand" RBSUnitExtensions to be able to deal with two SUnit implementations concurrently.

In the end, I decided to go on a journey. I forked the code. Not only SUnit. But RBSUnitExtensions as well. Again, not because I think mine is better or worse. It's different. In the Linux world, they have many different implementations of the same type of program. For varieties of reasons. This was done in the same spirit. I hope no one takes offense, none was intended.

!SUnitToo

SUnitToo has been published to the Open Repository. It is free, public domain, etc. It seems to work well under 7.2 and 7.2.1. It was derived from Niall Ross's latest 3.1 based SUnit packages (3.1 RC4). Points worth highlighting are:

• VisualWorks Specific - SUnitToo does not have a compatibility layer at all. It's all just VisualWorks stuff. It relies on the triggerEvent: system.
• One Package - There is just one Package. The Tests to test itself are in there. I know that some people like to separate tests from the code itself, so that they can deploy systems without the tests. Which is fine, but it doesn't really make sense in this case.
• Namespace - SUnitToo avoids conflict with SUnit original, by putting its classes in a separate namspace: Smalltalk.SUnit. This means that to use SUnitToo, you need to change the parent class of any of your test classes from XProgramming.SUnit.TestCase to just SUnit.TestCase.
• Resources - I don't really use these things. But I understand what they're for and many do use them. Consequently, this was one part of the system I left entire alone. I did my best to leave it exactly as is.
• Tests for Tests - The core of SUnit original is tested with one SUnitTests TestCase. I took a different tact and wrote a unit test per class. I like Unit tests. I like them alot. I think they benefit the design process greatly. In part, because they encourage me to write units (objects) that are more encapsulated than I might have other wise. The Resource tests were retained, the SUnitTest was replaced with 4 separate per object TestCases, comprising a total of 23 tests.
• Tear Down errors - When SUnit original runs, if it encounters an error in the tearDown method, it will report two results from the one test. This has been fixed.
• Memory Problem - This problem is fixed by the incorporation of a TestToken object. This object relieves TestCase of having to both be a token for which tests to run, or what the results are, and the actual fixture plus all of its state. TestSuites and TestResult model TestCases indirectly by retaining TestTokens instead. Originally, it was the plan to use CompiledMethods for this. But this creates a number of subtle problems, is kind of a hack, and isn't that efficient. Adding this simple object was a real boon. I was surprised there isn't something like this in the sytem. There is MethodDefinition, but it's a bit more than I needed (I did toy with using Points just for the grins of it).
• Pluggable via triggerEvents: - TestCase #run both returns the status of the test (#pass, #fail, or #error) as well as asks TestCase class to post an event as to the outcome of the TestCase instance. This is a fundamental shift from SUnit original. SUnit original, relies on the result to actually "interpret" the result of the test run. By moving it to the TestCase class, it allows tools to plug into these events and do whatever. In fact, TestResult itself is changed to behave this way. A TestResult is created which simply observes TestCase class for the duration of the run. This decouples the Result, Case, and Suite classes.
• Random Tests - A TestSuite when run, will always run its tests randomly. This doesn't just mean we use a Set. Access to the collection is truly randomized. This improves tests by decreasing test order dependency.
• Less Methods - I removed methods that were not used by the tools I'm familiar with pretty liberally. My take was, if I were designing SUnit today, in light of the "customer needs" of the tools, I would not include many of the "might want to do it this way" methods on the grounds that they would be YAGNI.
• Assert Variants - SUnit original has a number of assert: variants that allow for logging, as well as a continue on anyway. I did not retain these. It is my opinion, that the better mechanism for logging is to use something that plugs into the TestCase events. If there's a desire for an "assertButContinue" pattern, then a simple name should be created for that. I'm not sure what one reports for that kind failure...

!ExtraRBForSUnitToo

Once I gave up trying to make RBSUnitExtensions support two concurrent implementations, this freed me to have fun with a new version. It wouldn't be me unless I'd got another tool in with Extra in the name. So ExtraRBForSUnitToo was forked. It was derived from version 8 of RBSUnitExtensions. Notable differences are:

• Assert Debugs - When you debug a test, rather than just pop up an error notifier, which you then click debug on, and scroll backwards to find your testMethod, it actually opens the debbuger for you AND walks back in the context chain til it finds your method and highlights the failure point.
• Error Debugs - Errors open the debugger, rather than a notifier.
• Incremental Status - RBSUnitExtensions gives a Tests Remaining count as it counts down through tests. ExtraRBForSUnitToo prints an "n of total" counter, as well as the test name for each test as it runs through them. As well, as soon as a defect is detected, the field is turned red.
• Debug Updates Tests - Debug updates the incremental and final status the same as Run does.
• Hot Key - Well, I tried at least. For all I know, this actually works, but not on OSX because meta key handling for VW under OSX is about a stone's throw away from not at all.
• Categories - Any support for doing "category" based Tests was removed.

!Future Plans

I don't know yet what the future of SUnitToo is. It does provide a platform to do a VW specific version. It was in large an experiment, "Let's fork and see what we end up with." I will probably shift our systems to use it for the short term future. All of the outward changes could be rolled back into the existing SUnit and associated tools. For now, I need to get back to work.