Trait that defines an abstract patienceConfig
method that is implemented in PatienceConfiguration
and can
be overriden in stackable modification traits such as IntegrationPatience
.
Trait that facilitates performing assertions outside the main test thread, such as assertions in callback methods that are invoked asynchronously.
Trait that facilitates performing assertions outside the main test thread, such as assertions in callback methods that are invoked asynchronously.
Trait AsyncAssertions
provides a Waiter
class that you can use to orchestrate the inter-thread
communication required to perform assertions outside the main test thread, and a means to configure it.
To use Waiter
, create an instance of it in the main test thread:
val w = new Waiter // Do this in the main test thread
At some point later, call await
on the waiter:
w.await() // Call await() from the main test thread
The await
call will block until it either receives a report of a failed assertion from a different thread, at which
point it will complete abruptly with the same exception, or until it is dismissed by a different thread (or threads), at
which point it will return normally. You can optionally specify a timeout and/or a number
of dismissals to wait for. Here's an example:
import org.scalatest.time.SpanSugar._ w.await(timeout(300 millis), dismissals(2))
The default value for timeout
, provided via an implicit PatienceConfig
parameter, is 150 milliseconds. The default value for
dismissals
is 1. The await
method will block until either it is dismissed a sufficient number of times by other threads or
an assertion fails in another thread. Thus if you just want to perform assertions in just one other thread, only that thread will be
performing a dismissal, so you can use the default value of 1 for dismissals
.
Waiter
contains four overloaded forms of await
, two of which take an implicit
PatienceConfig
parameter. To change the default timeout configuration, override or hide
(if you imported the members of AsyncAssertions
companion object instead of mixing in the
trait) patienceConfig
with a new one that returns your desired configuration.
To dismiss a waiter, you just invoke dismiss
on it:
w.dismiss() // Call this from one or more other threads
You may want to put dismiss
invocations in a finally clause to ensure they happen even if an exception is thrown.
Otherwise if a dismissal is missed because of a thrown exception, an await
call will wait until it times out.
Note that if a Waiter
receives more than the expected number of dismissals, it will not report
this as an error: i.e., receiving greater than the number of expected dismissals without any failed assertion will simply
cause the the test to complete, not to fail. The only way a Waiter
will cause a test to fail is if one of the
asynchronous assertions to which it is applied fails.
Finally, to perform an assertion in a different thread, you just apply the Waiter
to the assertion code. Here are
some examples:
w { assert(1 + 1 === 3) } // Can use assertions w { 1 + 1 should equal (3) } // Or matchers w { "hi".charAt(-1) } // Any exceptions will be forwarded to await
Here's a complete example:
import org.scalatest._ import concurrent.AsyncAssertions import scala.actors.Actor class ExampleSuite extends FunSuite with Matchers with AsyncAssertions { case class Message(text: String) class Publisher extends Actor { @volatile private var handle: Message => Unit = { (msg) => } def registerHandler(f: Message => Unit) { handle = f } def act() { var done = false while (!done) { react { case msg: Message => handle(msg) case "Exit" => done = true } } } } test("example one") { val publisher = new Publisher val message = new Message("hi") val w = new Waiter publisher.start() publisher.registerHandler { msg => w { msg should equal (message) } w.dismiss() } publisher ! message w.await() publisher ! "Exit" } }
Trait that can pass a new Conductor
fixture into tests.
Trait that can pass a new Conductor
fixture into tests.
Here's an example of the use of this trait to test the ArrayBlockingQueue
class from java.util.concurrent
:
import org.scalatest.fixture import org.scalatest.concurrent.ConductorFixture import org.scalatest.matchers.ShouldMatchers import java.util.concurrent.ArrayBlockingQueue class ArrayBlockingQueueSuite extends fixture.FunSuite with ConductorFixture with ShouldMatchers { test("calling put on a full queue blocks the producer thread") { conductor => import conductor._ val buf = new ArrayBlockingQueue[Int](1) thread("producer") { buf put 42 buf put 17 beat should be (1) } thread("consumer") { waitForBeat(1) buf.take should be (42) buf.take should be (17) } whenFinished { buf should be ('empty) } } test("calling take on an empty queue blocks the consumer thread") { conductor => import conductor._ val buf = new ArrayBlockingQueue[Int](1) thread("producer") { waitForBeat(1) buf put 42 buf put 17 } thread("consumer") { buf.take should be (42) buf.take should be (17) beat should be (1) } whenFinished { buf should be ('empty) } } }
For an explanation of how these tests work, see the documentation for Conductors
.
Trait that provides each test with access to a new Conductor
via methods.
Trait that provides each test with access to a new Conductor
via methods.
Here's an example of the use of this trait to test the ArrayBlockingQueue
concurrency abstraction from java.util.concurrent
:
import org.scalatest.FunSuite import org.scalatest.concurrent.ConductorMethods import org.scalatest.matchers.ShouldMatchers import java.util.concurrent.ArrayBlockingQueue class ArrayBlockingQueueSuite extends FunSuite with ConductorMethods with ShouldMatchers { test("calling put on a full queue blocks the producer thread") { val buf = new ArrayBlockingQueue[Int](1) thread("producer") { buf put 42 buf put 17 beat should be (1) } thread("consumer") { waitForBeat(1) buf.take should be (42) buf.take should be (17) } whenFinished { buf should be ('empty) } } test("calling take on an empty queue blocks the consumer thread") { val buf = new ArrayBlockingQueue[Int](1) thread("producer") { waitForBeat(1) buf put 42 buf put 17 } thread("consumer") { buf.take should be (42) buf.take should be (17) beat should be (1) } whenFinished { buf should be ('empty) } } }
For an explanation of how these tests work, see the documentation for Conductors
.
Trait whose Conductor
member facilitates the testing of classes, traits, and libraries designed
to be used by multiple threads concurrently.
Trait whose Conductor
member facilitates the testing of classes, traits, and libraries designed
to be used by multiple threads concurrently.
A Conductor
conducts a multi-threaded scenario by maintaining
a clock of "beats." Beats are numbered starting with 0. You can ask a
Conductor
to run threads that interact with the class, trait,
or library (the subject)
you want to test. A thread can call the Conductor
's
waitForBeat
method, which will cause the thread to block
until that beat has been reached. The Conductor
will advance
the beat only when all threads participating in the test are blocked. By
tying the timing of thread activities to specific beats, you can write
tests for concurrent systems that have deterministic interleavings of
threads.
A Conductor
object has a three-phase lifecycle. It begins its life
in the setup phase. During this phase, you can start threads by
invoking the thread
method on the Conductor
.
When conduct
is invoked on a Conductor
, it enters
the conducting phase. During this phase it conducts the one multi-threaded
scenario it was designed to conduct. After all participating threads have exited, either by
returning normally or throwing an exception, the conduct
method
will complete, either by returning normally or throwing an exception. As soon as
the conduct
method completes, the Conductor
enters its defunct phase. Once the Conductor
has conducted
a multi-threaded scenario, it is defunct and can't be reused. To run the same test again,
you'll need to create a new instance of Conductor
.
Here's an example of the use of Conductor
to test the ArrayBlockingQueue
class from java.util.concurrent
:
import org.scalatest.fixture.FunSuite import org.scalatest.matchers.ShouldMatchers import java.util.concurrent.ArrayBlockingQueue import org.scalatest.concurrent.Conductors class ArrayBlockingQueueSuite extends FunSuite with ShouldMatchers with Conductors { test("calling put on a full queue blocks the producer thread") { val conductor = new Conductor import conductor._ val buf = new ArrayBlockingQueue[Int](1) thread("producer") { buf put 42 buf put 17 beat should be (1) } thread("consumer") { waitForBeat(1) buf.take should be (42) buf.take should be (17) } whenFinished { buf should be ('empty) } } }
When the test shown is run, it will create one thread named producer and another named
consumer. The producer thread will eventually execute the code passed as a by-name
parameter to thread("producer")
:
buf put 42 buf put 17 beat should be (1)
Similarly, the consumer thread will eventually execute the code passed as a by-name parameter
to thread("consumer")
:
waitForBeat(1) buf.take should be (42) buf.take should be (17)
The thread
creates the threads and starts them, but they will not immediately
execute the by-name parameter passed to them. They will first block, waiting for the Conductor
to give them a green light to proceed.
The next call in the test is whenFinished
. This method will first call conduct
on
the Conductor
, which will wait until all threads that were created (in this case, producer and consumer) are
at the "starting line", i.e., they have all started and are blocked, waiting on the green light.
The conduct
method will then give these threads the green light and they will
all start executing their blocks concurrently.
When the threads are given the green light, the beat is 0. The first thing the producer thread does is put 42 in
into the queue. As the queue is empty at this point, this succeeds. The producer thread next attempts to put a 17
into the queue, but because the queue has size 1, this can't succeed until the consumer thread has read the 42
from the queue. This hasn't happened yet, so producer blocks. Meanwhile, the consumer thread's first act is to
call waitForBeat(1)
. Because the beat starts out at 0, this call will block the consumer thread.
As a result, once the producer thread has executed buf put 17
and the consumer thread has executed
waitForBeat(1)
, both threads will be blocked.
The Conductor
maintains a clock that wakes up periodically and checks to see if all threads
participating in the multi-threaded scenario (in this case, producer and consumer) are blocked. If so, it
increments the beat. Thus sometime later the beat will be incremented, from 0 to 1. Because consumer was
waiting for beat 1, it will wake up (i.e., the waitForBeat(1)
call will return) and
execute the next line of code in its block, buf.take should be (42)
. This will succeed, because
the producer thread had previously (during beat 0) put 42 into the queue. This act will also make
producer runnable again, because it was blocked on the second put
, which was waiting for another
thread to read that 42.
Now both threads are unblocked and able to execute their next statement. The order is
non-deterministic, and can even be simultaneous if running on multiple cores. If the consumer
thread
happens to execute buf.take should be (17)
first, it will block (buf.take
will not return), because the queue is
at that point empty. At some point later, the producer thread will execute buf put 17
, which will
unblock the consumer thread. Again both threads will be runnable and the order non-deterministic and
possibly simulataneous. The producer thread may charge ahead and run its next statement, beat should be (1)
.
This will succeed because the beat is indeed 1 at this point. As this is the last statement in the producer's block,
the producer thread will exit normally (it won't throw an exception). At some point later the consumer thread will
be allowed to complete its last statement, the buf.take
call will return 17. The consumer thread will
execute 17 should be (17)
. This will succeed and as this was the last statement in its block, the consumer will return
normally.
If either the producer or consumer thread had completed abruptbly with an exception, the conduct
method
(which was called by whenFinished
) would have completed abruptly with an exception to indicate the test
failed. However, since both threads returned normally, conduct
will return. Because conduct
doesn't
throw an exception, whenFinished
will execute the block of code passed as a by-name parameter to it: buf should be ('empty)
.
This will succeed, because the queue is indeed empty at this point. The whenFinished
method will then return, and
because the whenFinished
call was the last statement in the test and it didn't throw an exception, the test completes successfully.
This test tests ArrayBlockingQueue
, to make sure it works as expected. If there were a bug in ArrayBlockingQueue
such as a put
called on a full queue didn't block, but instead overwrote the previous value, this test would detect
it. However, if there were a bug in ArrayBlockingQueue
such that a call to take
called on an empty queue
never blocked and always returned 0, this test might not detect it. The reason is that whether the consumer thread will ever call
take
on an empty queue during this test is non-deterministic. It depends on how the threads get scheduled during beat 1.
What is deterministic in this test, because the consumer thread blocks during beat 0, is that the producer thread will definitely
attempt to write to a full queue. To make sure the other scenario is tested, you'd need a different test:
test("calling take on an empty queue blocks the consumer thread") { val conductor = new Conductor import conductor._ val buf = new ArrayBlockingQueue[Int](1) thread("producer") { waitForBeat(1) buf put 42 buf put 17 } thread("consumer") { buf.take should be (42) buf.take should be (17) beat should be (1) } whenFinished { buf should be ('empty) } }
In this test, the producer thread will block, waiting for beat 1. The consumer thread will invoke buf.take
as its first act. This will block, because the queue is empty. Because both threads are blocked, the Conductor
will at some point later increment the beat to 1. This will awaken the producer thread. It will return from its
waitForBeat(1)
call and execute buf put 42
. This will unblock the consumer thread, which will
take the 42, and so on.
The problem that Conductor
is designed to address is the difficulty, caused by the non-deterministic nature
of thread scheduling, of testing classes, traits, and libraries that are intended to be used by multiple threads.
If you just create a test in which one thread reads from an ArrayBlockingQueue
and
another writes to it, you can't be sure that you have tested all possible interleavings of threads, no matter
how many times you run the test. The purpose of Conductor
is to enable you to write tests with deterministic interleavings of threads. If you write one test for each possible
interleaving of threads, then you can be sure you have all the scenarios tested. The two tests shown here, for example,
ensure that both the scenario in which a producer thread tries to write to a full queue and the scenario in which a
consumer thread tries to take from an empty queue are tested.
Class Conductor
was inspired by the
MultithreadedTC project,
created by Bill Pugh and Nat Ayewah of the University of Maryland.
Although useful, bear in mind that a Conductor
's results are not guaranteed to be
accurate 100% of the time. The reason is that it uses java.lang.Thread
's getState
method to
decide when to advance the beat. This use goes against the advice given in the Javadoc documentation for
getState
, which says, "This method is designed for use in monitoring of the system state, not for
synchronization." In short, sometimes the return value of getState
occasionally may be inacurrate,
which in turn means that sometimes a Conductor
could decide to advance the beat too early. In practice,
Conductor
has proven to be very helpful when developing thread safe classes. It is also useful in
for regression tests, but you may have to tolerate occasional false negatives.
Trait that provides the eventually
construct, which periodically retries executing
a passed by-name parameter, until it either succeeds or the configured timeout has been surpassed.
Trait that provides the eventually
construct, which periodically retries executing
a passed by-name parameter, until it either succeeds or the configured timeout has been surpassed.
The by-name parameter "succeeds" if it returns a result. It "fails" if it throws any exception that
would normally cause a test to fail. (These are any exceptions except TestPendingException
and
Error
s listed in the
Treatment of java.lang.Error
s section of the
documentation of trait Suite
.)
For example, the following invocation of eventually
would succeed (not throw an exception):
val xs = 1 to 125 val it = xs.iterator eventually { it.next should be (3) }
However, because the default timeout is 150 milliseconds, the following invocation of
eventually
would ultimately produce a TestFailedDueToTimeoutException
:
val xs = 1 to 125 val it = xs.iterator eventually { Thread.sleep(50); it.next should be (110) }
Assuming the default configuration parameters, a timeout
of 150 milliseconds and an interval
of 15 milliseconds,
were passed implicitly to eventually
, the detail message of the thrown
TestFailedDueToTimeoutException
would look like:
The code passed to eventually never returned normally. Attempted 2 times over 166.682 milliseconds. Last failure message: 2 was not equal to 110.
The cause of the thrown TestFailedDueToTimeoutException
will be the exception most recently thrown by the block of code passed to eventually. (In
the previous example, the cause would be the TestFailedException
with the detail message 2 was not equal to 100
.)
eventually
The eventually
methods of this trait can be flexibly configured.
The two configuration parameters for eventually
along with their
default values and meanings are described in the following table:
Configuration Parameter | Default Value | Meaning |
---|---|---|
timeout
|
scaled(150 milliseconds)
|
the maximum amount of time to allow unsuccessful attempts before giving up and throwing TestFailedDueToTimeoutException
|
interval
|
scaled(15 milliseconds)
|
the amount of time to sleep between each attempt |
The default values of both timeout and interval are passed to the scaled
method, inherited
from ScaledTimeSpans
, so that the defaults can be scaled up
or down together with other scaled time spans. See the documentation for trait ScaledTimeSpans
for more information.
The eventually
methods of trait Eventually
each take a PatienceConfig
object as an implicit parameter. This object provides values for the two configuration parameters. (These configuration parameters
are called "patience" because they determine how patient tests will be with asynchronous operations: how long
they will tolerate failures before giving up and how long they will wait before checking again after a failure.) Trait
Eventually
provides an implicit val
named patienceConfig
with each
configuration parameter set to its default value.
If you want to set one or more configuration parameters to a different value for all invocations of
eventually
in a suite you can override this
val (or hide it, for example, if you are importing the members of the Eventually
companion object rather
than mixing in the trait). For example, if
you always want the default timeout
to be 2 seconds and the default interval
to be 5 milliseconds, you
can override patienceConfig
, like this:
implicit override val patienceConfig = PatienceConfig(timeout = scaled(Span(2, Seconds)), interval = scaled(Span(5, Millis)))
Or, hide it by declaring a variable of the same name in whatever scope you want the changed values to be in effect:
implicit val patienceConfig = PatienceConfig(timeout = scaled(Span(2, Seconds)), interval = scaled(Span(5, Millis)))
Passing your new default values to scaled
is optional, but a good idea because it allows them to
be easily scaled if run on a slower or faster system.
In addition to taking a PatienceConfig
object as an implicit parameter, the eventually
methods of trait
Eventually
include overloaded forms that take one or two PatienceConfigParam
objects that you can use to override the values provided by the implicit PatienceConfig
for a single eventually
invocation. For example, if you want to set timeout
to 5 seconds for just one particular eventually
invocation,
you can do so like this:
eventually (timeout(Span(5, Seconds))) { Thread.sleep(10); it.next should be (110) }
This invocation of eventually
will use 5 seconds for the timeout
and whatever value is specified by the
implicitly passed PatienceConfig
object for the interval
configuration parameter.
If you want to set both configuration parameters in this way, just list them separated by commas:
eventually (timeout(Span(5, Seconds)), interval(Span(5, Millis))) { it.next should be (110) }
You can also import or mix in the members of SpanSugar
if
you want a more concise DSL for expressing time spans:
eventually (timeout(5 seconds), interval(5 millis)) { it.next should be (110) }
Note that ScalaTest will not scale any time span that is not explicitly passed to scaled
to make
the meaning of the code as obvious as possible. Thus
if you ask for "timeout(5 seconds)
" you will get exactly that: a timeout of five seconds. If you want such explicitly
given values to be scaled, you must pass them to scale
explicitly like this:
eventually (timeout(scaled(5 seconds))) { it.next should be (110) }
The previous code says more clearly that the timeout will be five seconds, unless scaled higher or lower by the scaled
method.
The eventually
methods employ a very simple backoff algorithm to try and maximize the speed of tests. If an asynchronous operation
completes quickly, a smaller interval will yield a faster test. But if an asynchronous operation takes a while, a small interval will keep the CPU
busy repeatedly checking and rechecking a not-ready operation, to some extent taking CPU cycles away from other processes that could proceed. To
strike the right balance between these design tradeoffs, the eventually
methods will check more frequently during the initial interval.
Rather than sleeping an entire interval if the initial attempt fails, eventually
will only sleep 1/10 of the configured interval. It
will continue sleeping only 1/10 of the configured interval until the configured interval has passed, after which it sleeps the configured interval
between attempts. Here's an example in which the timeout is set equal to the interval:
val xs = 1 to 125 val it = xs.iterator eventually(timeout(100 milliseconds), interval(100 milliseconds)) { it.next should be (110) }
Even though this call to eventually
will time out after only one interval, approximately, the error message will likely report that more
than one (and less than ten) attempts were made:
The code passed to eventually never returned normally. Attempted 6 times over 100.485 milliseconds. Last failure message: 6 was not equal to 110.
Note that if the initial attempt takes longer than the configured interval to complete, eventually
will never sleep for
a 1/10 interval. You can observe this behavior in the second example above in which the first statement in the block of code passed to eventually
was Thread.sleep(50)
.
Eventually
intended primarily for integration testing Although the default timeouts of trait Eventually
are tuned for unit testing, the use of Eventually
in unit tests is
a choice you should question. Usually during unit testing you'll want to mock out subsystems that would require Eventually
, such as
network services with varying and unpredictable response times. This will allow your unit tests to run as fast as possible while still testing
the focused bits of behavior they are designed to test.
Nevertheless, because sometimes it will make sense to use Eventually
in unit tests (and
because it is destined to happen anyway even when it isn't the best choice), Eventually
by default uses
timeouts tuned for unit tests: Calls to eventually
are more likely to succeed on fast development machines, and if a call does time out,
it will do so quickly so the unit tests can move on.
When you are using Eventually
for integration testing, therefore, the default timeout and interval may be too small. A
good way to override them is by mixing in trait IntegrationPatience
or a similar trait of your
own making. Here's an example:
class ExampleSpec extends FeatureSpec with Eventually with IntegrationPatience { // Your integration tests here... }
Trait IntegrationPatience
increases the default timeout from 150 milliseconds to 15 seconds, the default
interval from 15 milliseconds to 150 milliseconds. If need be, you can do fine tuning of the timeout and interval by
specifying a time span scale factor when you
run your tests.
Trait that facilitates testing with futures.
Trait that facilitates testing with futures.
This trait defines a FutureConcept
trait that can be used to implicitly wrap
different kinds of futures, thereby providing a uniform testing API for futures.
The three ways this trait enables you to test futures are:
1. Invoking isReadyWithin
, to assert that a future is ready within a a specified time period.
Here's an example:
assert(result.isReadyWithin(100 millis))
2. Invoking futureValue
, to obtain a futures result within a specified or implicit time period,
like this:
assert(result.futureValue === 7)
3. Passing the future to whenReady
, and performing assertions on the result value passed
to the given function, as in:
whenReady(result) { s => s should be ("hello") }
The whenReady
construct periodically inspects the passed
future, until it is either ready or the configured timeout has been surpassed. If the future becomes
ready before the timeout, whenReady
passes the future's value to the specified function.
To make whenReady
more broadly applicable, the type of future it accepts is a FutureConcept[T]
,
where T
is the type of value promised by the future. Passing a future to whenReady
requires
an implicit conversion from the type of future you wish to pass (the modeled type) to
FutureConcept[T]
. Subtrait JavaFutures
provides an implicit conversion from
java.util.concurrent.Future[T]
to FutureConcept[T]
.
For example, the following invocation of whenReady
would succeed (not throw an exception):
import org.scalatest._ import matchers.ShouldMatchers._ import concurrent.Futures._ import java.util.concurrent._ val exec = Executors.newSingleThreadExecutor val task = new Callable[String] { def call() = { Thread.sleep(50); "hi" } } whenReady(exec.submit(task)) { s => s should be ("hi") }
However, because the default timeout is 150 milliseconds, the following invocation of
whenReady
would ultimately produce a TestFailedException
:
val task = new Callable[String] { def call() = { Thread.sleep(500); "hi" } } whenReady(exec.submit(task)) { s => s should be ("hi") }
Assuming the default configuration parameters, a timeout
of 150 milliseconds and an
interval
of 15 milliseconds,
were passed implicitly to whenReady
, the detail message of the thrown
TestFailedException
would look like:
The future passed to whenReady was never ready, so whenReady timed out. Queried 95 times, sleeping 10 milliseconds between each query.
whenReady
The whenReady
methods of this trait can be flexibly configured.
The two configuration parameters for whenReady
along with their
default values and meanings are described in the following table:
Configuration Parameter | Default Value | Meaning |
---|---|---|
timeout | scaled(150 milliseconds) |
the maximum amount of time to allow unsuccessful queries before giving up and throwing TestFailedException
|
interval | scaled(15 milliseconds) | the amount of time to sleep between each query |
The default values of both timeout and interval are passed to the scaled
method, inherited
from ScaledTimeSpans
, so that the defaults can be scaled up
or down together with other scaled time spans. See the documentation for trait ScaledTimeSpans
for more information.
The whenReady
methods of trait Futures
each take a PatienceConfig
object as an implicit parameter. This object provides values for the two configuration parameters. Trait
Futures
provides an implicit val
named defaultPatience
with each
configuration parameter set to its default value.
If you want to set one or more configuration parameters to a different value for all invocations of
whenReady
in a suite you can override this
val (or hide it, for example, if you are importing the members of the Futures
companion object rather
than mixing in the trait). For example, if
you always want the default timeout
to be 2 seconds and the default interval
to be 5 milliseconds, you
can override defaultPatience
, like this:
implicit override val defaultPatience = PatienceConfig(timeout = Span(2, Seconds), interval = Span(5, Millis))
Or, hide it by declaring a variable of the same name in whatever scope you want the changed values to be in effect:
implicit val defaultPatience = PatienceConfig(timeout = Span(2, Seconds), interval = Span(5, Millis))
In addition to taking a PatienceConfig
object as an implicit parameter, the whenReady
methods of trait
Futures
include overloaded forms that take one or two PatienceConfigParam
objects that you can use to override the values provided by the implicit PatienceConfig
for a single whenReady
invocation. For example, if you want to set timeout
to 6 seconds for just one particular whenReady
invocation,
you can do so like this:
whenReady (exec.submit(task), timeout(Span(6, Seconds))) { s => s should be ("hi") }
This invocation of eventually
will use 6000 for timeout
and whatever value is specified by the
implicitly passed PatienceConfig
object for the interval
configuration parameter.
If you want to set both configuration parameters in this way, just list them separated by commas:
whenReady (exec.submit(task), timeout(Span(6, Seconds)), interval(Span(500, Millis))) { s => s should be ("hi") }
You can also import or mix in the members of SpanSugar
if
you want a more concise DSL for expressing time spans:
whenReady (exec.submit(task), timeout(6 seconds), interval(500 millis)) { s => s should be ("hi") }
Note: The whenReady
construct was in part inspired by the whenDelivered
matcher of the
BlueEyes project, a lightweight, asynchronous web framework for Scala.
Stackable modification trait for PatienceConfiguration
that provides default timeout and interval
values appropriate for integration testing.
Stackable modification trait for PatienceConfiguration
that provides default timeout and interval
values appropriate for integration testing.
The default values for the parameters are:
Configuration Parameter | Default Value |
---|---|
timeout
|
scaled(15 seconds)
|
interval
|
scaled(150 milliseconds)
|
The default values of both timeout and interval are passed to the scaled
method, inherited
from ScaledTimeSpans
, so that the defaults can be scaled up
or down together with other scaled time spans. See the documentation for trait ScaledTimeSpans
for more information.
Mix this trait into any class that uses PatienceConfiguration
(such as classes that mix in Eventually
or AsyncAssertions
) to get timeouts tuned towards integration testing, like this:
class ExampleSpec extends FeatureSpec with Eventually with IntegrationPatience { // ... }
Strategy for interrupting an operation after a timeout expires.
Strategy for interrupting an operation after a timeout expires.
An instance of this trait is used for configuration when using traits
Timeouts
and TimeLimitedTests
.
Provides an implicit conversion from java.util.concurrent.Future[T]
to
FutureConcept[T]
.
Provides an implicit conversion from java.util.concurrent.Future[T]
to
FutureConcept[T]
.
This trait enables you to invoke the methods defined on FutureConcept
on a Java Future
, as well as to pass a Java future
to the whenReady
methods of supertrait Futures
.
See the documentation for supertrait Futures
for the details on the syntax this trait provides
for testing with Java futures.
Trait providing methods and classes used to configure timeouts and, where relevant, the interval between retries.
Trait providing methods and classes used to configure timeouts and, where relevant, the interval between retries.
This trait is called PatienceConfiguration
because it allows configuration of two
values related to patience: The timeout specifies how much time asynchronous operations will be given
to succeed before giving up. The interval specifies how much time to wait between checks to determine
success when polling.
The default values for timeout and interval provided by trait PatienceConfiguration
are tuned for unit testing,
where running tests as fast as
possible is a high priority and subsystems requiring asynchronous operations are therefore often replaced
by mocks. This table shows the default values:
Configuration Parameter | Default Value |
---|---|
timeout
|
scaled(150 milliseconds)
|
interval
|
scaled(15 milliseconds)
|
Values more appropriate to integration testing, where asynchronous operations tend to take longer because the tests are run
against the actual subsytems (not mocks), can be obtained by mixing in trait IntegrationPatience
.
The default values of both timeout and interval are passed to the scaled
method, inherited
from ScaledTimeSpans
, so that the defaults can be scaled up
or down together with other scaled time spans. See the documentation for trait ScaledTimeSpans
for more information.
Timeouts are used by the eventually
methods of trait
Eventually
and the await
method of class
Waiter
, a member of trait
AsyncAssertions
. Intervals are used by
the eventually
methods.
Provides an implicit conversion from scala.concurrent.Future[T]
to
FutureConcept[T]
.
Provides an implicit conversion from scala.concurrent.Future[T]
to
FutureConcept[T]
.
This trait enables you to invoke the methods defined on FutureConcept
on a Scala Future
, as well as to pass a Scala future
to the whenReady
methods of supertrait Futures
.
See the documentation for supertrait Futures
for the details on the syntax this trait provides
for testing with Scala futures.
Trait providing a scaled
method that can be used to scale time
Span
s used during the testing of asynchronous operations.
Trait providing a scaled
method that can be used to scale time
Span
s used during the testing of asynchronous operations.
The scaled
method allows tests of asynchronous operations to be tuned
according to need. For example, Span
s can be scaled larger when running
tests on slower continuous integration servers or smaller when running on faster
development machines.
The Double
factor by which to scale the Span
s passed to
scaled
is obtained from the spanScaleFactor
method, also declared
in this trait. By default this method returns 1.0, but can be configured to return
a different value by passing a -F
argument to Runner
(or
an equivalent mechanism in an ant, sbt, or Maven build file).
The default timeouts and intervals defined for traits Eventually
and
AsyncAssertions
invoke scaled
, so those defaults
will be scaled automatically. Other than such defaults, however, to get a Span
to scale you'll need to explicitly pass it to scaled
.
For example, here's how you would scale a Span
you supply to
the failAfter
method from trait Timeouts
:
failAfter(scaled(150 millis)) { // ... }
The reason Span
s are not scaled automatically in the general case is
to make code obvious. If a reader sees failAfter(1 second)
, it will
mean exactly that: fail after one second. And if a Span
will be scaled,
the reader will clearly see that as well: failAfter(scaled(1 second))
.
spanScaleFactor
You can override the spanScaleFactor
method to configure the factor by a
different means. For example, to configure the factor from Akka
TestKit's test time factor you might create a trait like this:
import org.scalatest.concurrent.ScaledTimeSpans import akka.actor.ActorSystem import akka.testkit.TestKitExtension trait AkkaSpanScaleFactor extends ScaledTimeSpans { override def spanScaleFactor: Double = TestKitExtension.get(ActorSystem()).TestTimeFactor }
This trait overrides spanScaleFactor
so that it takes its
scale factor from Akka's application.conf
file.
You could then scale Span
s tenfold in Akka's configuration file
like this:
akka { test { timefactor = 10.0 } }
Armed with this trait and configuration file, you can simply mix trait
AkkaSpanScaleFactor
into any test class whose Span
s
you want to scale, like this:
class MySpec extends FunSpec with Eventually with AkkaSpanScaleFactor { // .. }
Strategy for interrupting an operation in which wakeup
is called on the java.nio.channels.Selector
passed to
the constructor.
Strategy for interrupting an operation in which wakeup
is called on the java.nio.channels.Selector
passed to
the constructor.
This class can be used for configuration when using traits Timeouts
and TimeLimitedTests
.
Strategy for interrupting an operation in which close
is called on the java.net.Socket
passed to
the constructor.
Strategy for interrupting an operation in which close
is called on the java.net.Socket
passed to
the constructor.
This class can be used for configuration when using traits Timeouts
and TimeLimitedTests
.
Trait that when mixed into a suite class establishes a time limit for its tests.
Trait that when mixed into a suite class establishes a time limit for its tests.
This trait overrides withFixture
, wrapping a super.withFixture(test)
call
in a failAfter
invocation, specifying a timeout obtained by invoking timeLimit
and an Interruptor
by invoking defaultTestInterruptor
:
failAfter(timeLimit) { super.withFixture(test) } (defaultTestInterruptor)
Note that the failAfter
method executes the body of the by-name passed to it using the same
thread that invoked failAfter
. This means that the same thread will run the withFixture
method
as well as each test, so no extra synchronization is required. A second thread is used to run a timer, and if the timeout
expires, that second thread will attempt to interrupt the main test thread via the defaultTestInterruptor
.
The timeLimit
field is abstract in this trait. Thus you must specify a time limit when you use it.
For example, the following code specifies that each test must complete within 200 milliseconds:
import org.scalatest.FunSpec import org.scalatest.concurrent.TimeLimitedTests import org.scalatest.time.SpanSugar._ class ExampleSpec extends FunSpec with TimeLimitedTests { // Note: You may need to either write 200.millis or (200 millis), or // place a semicolon or blank line after plain old 200 millis, to // avoid the semicolon inference problems of postfix operator notation. val timeLimit = 200 millis describe("A time-limited test") { it("should succeed if it completes within the time limit") { Thread.sleep(100) } it("should fail if it is taking too darn long") { Thread.sleep(300) } } }
If you run the above ExampleSpec
, the second test will fail with the error message:
The test did not complete within the specified 200 millisecond time limit.
The failAfter
method uses an Interruptor
to attempt to interrupt the main test thread if the timeout
expires. The default Interruptor
returned by the defaultTestInterruptor
method is a
ThreadInterruptor
, which calls interrupt
on the main test thread. If you wish to change this
interruption strategy, override defaultTestInterruptor
to return a different Interruptor
. For example,
here's how you'd change the default to DoNotInterrupt
, a very patient interruption strategy that does nothing to
interrupt the main test thread:
import org.scalatest.FunSpec import org.scalatest.concurrent.TimeLimitedTests import org.scalatest.time.SpanSugar._ class ExampleSpec extends FunSpec with TimeLimitedTests { val timeLimit = 200 millis override val defaultTestInterruptor = DoNotInterrupt describe("A time-limited test") { it("should succeed if it completes within the time limit") { Thread.sleep(100) } it("should fail if it is taking too darn long") { Thread.sleep(300) } } }
Like the previous incarnation of ExampleSuite
, the second test will fail with an error message that indicates
a timeout expired. But whereas in the previous case, the Thread.sleep
would be interrupted after 200 milliseconds,
in this case it is never interrupted. In the previous case, the failed test requires a little over 200 milliseconds to run.
In this case, because the sleep(300)
is never interrupted, the failed test requires a little over 300 milliseconds
to run.
Trait that provides a failAfter
and cancelAfter
construct, which allows you to specify a time limit for an
operation passed as a by-name parameter, as well as a way to interrupt it if the operation exceeds its time limit.
Trait that provides a failAfter
and cancelAfter
construct, which allows you to specify a time limit for an
operation passed as a by-name parameter, as well as a way to interrupt it if the operation exceeds its time limit.
The time limit is passed as the first parameter, as a Span
. The operation is
passed as the second parameter. And an Interruptor
, a strategy for interrupting the operation, is
passed as an implicit third parameter. Here's a simple example of its use:
failAfter(Span(100, Millis)) { Thread.sleep(200) }
The above code, after 100 milliseconds, will produce a TestFailedDueToTimeoutException
with a message
that indicates a timeout expired:
The code passed to failAfter did not complete within 100 milliseconds.
If you use cancelAfter
in place of failAfter
, a TestCanceledException
with a message
that indicates a timeout expired:
The code passed to cancelAfter did not complete within 100 milliseconds.
If you prefer you can mix in or import the members of SpanSugar
and place a units value after the integer timeout.
Here are some examples:
import org.scalatest.time.SpanSugar._ failAfter(100 millis) { Thread.sleep(200) } failAfter(1 second) { Thread.sleep(2000) }
The code passed via the by-name parameter to failAfter
or cancelAfter
will be executed by the thread that invoked
failAfter
or cancelAfter
, so that no synchronization is necessary to access variables declared outside the by-name.
var result = -1 // No need to make this volatile failAfter(100 millis) { result = accessNetService() } result should be (99)
The failAfter
or cancelAfter
method will create a timer that runs on a different thread than the thread that
invoked failAfter
or cancelAfter
, so that it can detect when the timeout has expired and attempt to interrupt
the main thread. Because different operations can require different interruption strategies, the failAfter
or cancelAfter
method accepts an implicit third parameter of type Interruptor
that is responsible for interrupting
the main thread.
failAfter
or cancelAfter
with an Interruptor
This trait declares an implicit val
named defaultInterruptor
,
initialized with a ThreadInterruptor
, which attempts to interrupt the main thread by invoking
Thread.interrupt
. If you wish to use a different strategy, you can override this val
(or hide
it, for example if you imported the members of Timeouts
rather than mixing it in). Here's an example
in which the default interruption method is changed to DoNotInterrupt
, which does not attempt to
interrupt the main thread in any way:
override val defaultInterruptor = DoNotInterrupt failAfter(100 millis) { Thread.sleep(500) }
As with the default Interruptor
, the above code will eventually produce a
TestFailedDueToTimeoutException
with a message that indicates a timeout expired. However, instead
of throwing the exception after approximately 100 milliseconds, it will throw it after approximately 500 milliseconds.
This illustrates an important feature of failAfter
and cancelAfter
: it will throw a
TestFailedDueToTimeoutException
(or TestCanceledException
in case of cancelAfter
)
if the code passed as the by-name parameter takes longer than the specified timeout to execute, even if it
is allowed to run to completion beyond the specified timeout and returns normally.
ScalaTest provides the following Interruptor
implementations:
Interruptor implementation
|
Usage |
---|---|
ThreadInterruptor |
The default interruptor, invokes interrupt on the main test thread. This will
set the interrupted status for the main test thread and,
if the main thread is blocked, will in some cases cause the main thread to complete abruptly with
an InterruptedException .
|
DoNotInterrupt | Does not attempt to interrupt the main test thread in any way |
SelectorInterruptor |
Invokes wakeup on the passed java.nio.channels.Selector , which
will cause the main thread, if blocked in Selector.select , to complete abruptly with a
ClosedSelectorException .
|
SocketInterruptor |
Invokes close on the java.io.Socket , which
will cause the main thread, if blocked in a read or write of an java.io.InputStream or
java.io.OutputStream that uses the Socket , to complete abruptly with a
SocketException .
|
You may wish to create your own Interruptor
in some situations. For example, if your operation is performing
a loop and can check a volatile flag each pass through the loop. You could in that case write an Interruptor
that
sets that flag so that the next time around, the loop would exit.
Companion object that facilitates the importing of AsyncAssertions
members as
an alternative to mixing in the trait.
Companion object that facilitates the importing of AsyncAssertions
members as
an alternative to mixing in the trait. One use case is to import AsyncAssertions
's members so you can use
them in the Scala interpreter.
Interruption strategy in which nothing is done to try and interrupt an operation.
Interruption strategy in which nothing is done to try and interrupt an operation.
This object can be used for configuration when using traits Timeouts
and TimeLimitedTests
.
Companion object that facilitates the importing of Eventually
members as
an alternative to mixing in the trait.
Companion object that facilitates the importing of Eventually
members as
an alternative to mixing in the trait. One use case is to import Eventually
's members so you can use
them in the Scala interpreter:
$ scala -cp scalatest-1.8.jar Welcome to Scala version 2.9.1.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_29). Type in expressions to have them evaluated. Type :help for more information. scala> import org.scalatest._ import org.scalatest._ scala> import matchers.ShouldMatchers._ import matchers.ShouldMatchers._ scala> import concurrent.Eventually._ import concurrent.Eventually._ scala> val xs = 1 to 125 xs: scala.collection.immutable.Range.Inclusive = Range(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, ..., 125) scala> val it = xs.iterator it: Iterator[Int] = non-empty iterator scala> eventually { it.next should be (3) } scala> eventually { Thread.sleep(999); it.next should be (3) } org.scalatest.TestFailedException: The code passed to eventually never returned normally. Attempted 2 times, sleeping 10 milliseconds between each attempt. at org.scalatest.Eventually$class.tryTryAgain$1(Eventually.scala:313) at org.scalatest.Eventually$class.eventually(Eventually.scala:322) ...
Companion object that provides a factory method for an Interruptor
defined
in terms of a function from a function of type Thread
to Unit.
Companion object that provides a factory method for an Interruptor
defined
in terms of a function from a function of type Thread
to Unit.
Companion object that facilitates the importing of ScalaFutures
members as
an alternative to mixing in the trait.
Companion object that facilitates the importing of ScalaFutures
members as
an alternative to mixing in the trait. One use case is to import ScalaFutures
's members so you can use
them in the Scala interpreter.
Companion object that provides a factory method for a SelectorInterruptor
.
Companion object that provides a factory method for a SelectorInterruptor
.
Companion object that provides a factory method for a SocketInterruptor
.
Companion object that provides a factory method for a SocketInterruptor
.
Strategy for interrupting an operation in which interrupt
is called on the Thread
passed
to apply
.
Strategy for interrupting an operation in which interrupt
is called on the Thread
passed
to apply
.
This object can be used for configuration when using traits Timeouts
and TimeLimitedTests
.
Companion object that facilitates the importing of Timeouts
members as
an alternative to mixing in the trait.
Companion object that facilitates the importing of Timeouts
members as
an alternative to mixing in the trait. One use case is to import Timeouts
's members so you can use
them in the Scala interpreter.
Trait that defines an abstract
patienceConfig
method that is implemented inPatienceConfiguration
and can be overriden in stackable modification traits such asIntegrationPatience
.The main purpose of
AbstractPatienceConfiguration
is to differentiate corePatienceConfiguration
traits, such asEventually
andAsyncAssertions
, from stackable modification traits forPatienceConfiguration
s such asIntegrationPatience
. Because these stackable traits extendAbstractPatienceConfiguration
instead ofSuite
, you can't simply mix in a stackable trait:The previous code is undesirable because
IntegrationPatience
would have no affect on the class. Instead, you need to mix in a corePatienceConfiguration
trait and mix the stackableIntegrationPatience
trait into that, like this:The previous code is better because
IntegrationPatience
does have an effect: it modifies the behavior ofEventually
.