The Artima Developer Community
Sponsored Link

Ruby Buzz Forum
Functional Test Matrix, cont. (sorta)

0 replies on 1 page.

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 0 replies on 1 page
Ryan Davis

Posts: 651
Nickname: zenspider
Registered: Oct, 2004

Ryan Davis is a ruby nerd.
Functional Test Matrix, cont. (sorta) Posted: Apr 10, 2007 2:21 AM
Reply to this message Reply

This post originated from an RSS feed registered with Ruby Buzz by Ryan Davis.
Original Post: Functional Test Matrix, cont. (sorta)
Feed Title: Polishing Ruby
Feed URL: http://blog.zenspider.com/index.rdf
Feed Description: Musings on Ruby and the Ruby Community...
Latest Ruby Buzz Posts
Latest Ruby Buzz Posts by Ryan Davis
Latest Posts From Polishing Ruby

Advertisement

I've made some really good progress with the Functional Test Matrix idea. The code compression ratio is phenomenal. I'm writing up a comparison of the two, but without finesse the post would be huge and boring. I'll summarize here as best as I can.

I think most important idea is that high-level reviews or even edits of the functional tests require reading 1 line per action instead of N average lines per action * M edge cases. Very rough estimates show that my traditional functional test writing style weighs in at 179 lines of test for two actions against the following matrix:

setups         :u_w,   :u_r,  :g_w,  :g_r,  :o_w,  :o_W,  :o_r,  :o_R
matrix :edit, :edit,  :edit, :edit, :edit, :e_RO, :e_RO, :e_RO, :e_RO
matrix :view, :view,  :view, :view, :view, :view, :view, :view, :e_NF

Compare that to the 3 lines above and an additional 61 lines of "legos" used to put the matrix together (and, if you want, another 17 lines for the matrix framework itself). The liberal estimate puts that at 2.8:1, and the conservative estimate at 2.2:1. I suspect these numbers to be a bit high and for it to probably move down closer to 2.0:1.

The kicker for me tho, is that once the testing matures, I think we actually settle on the ideal of only really working with the matrix itself, pushing us up to nearly a 60:1 ratio! Like I said, this is rough, so YMMV. Look above. When my client comes to me and says "I think it'd be more secure all read-only errors were not-found errors", I wind up modifying only 4 result codes. If we change our permissions model and add unreadable and unwritable to group, I add 4 result codes and get 4 more tests for free.

A side benefit (from my perspective) is that the functional test matrix itself is high-level enough that it can be read and understood by the business owner or domain analyst. Allowing them to glance at it and immediately point out either missing edge cases or incorrect results.

Read: Functional Test Matrix, cont. (sorta)

Topic: Building a del.ico.us and flickr sidebar in 5 minutes Previous Topic   Next Topic Topic: Bill O'Reilly in St. Louis

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use