Class that facilitates the testing of classes, traits, and libraries designed to be used by multiple threads concurrently.
Class that facilitates the testing of classes, traits, and libraries designed to be used by multiple threads concurrently.
A Conductor
conducts a multi-threaded scenario by maintaining
a clock of "beats." Beats are numbered starting with 0. You can ask a
Conductor
to run threads that interact with the class, trait,
or library (the subject)
you want to test. A thread can call the Conductor
's
waitForBeat
method, which will cause the thread to block
until that beat has been reached. The Conductor
will advance
the beat only when all threads participating in the test are blocked. By
tying the timing of thread activities to specific beats, you can write
tests for concurrent systems that have deterministic interleavings of
threads.
A Conductor
object has a three-phase lifecycle. It begins its life
in the setup phase. During this phase, you can start threads by
invoking the thread
method on the Conductor
.
When conduct
is invoked on a Conductor
, it enters
the conducting phase. During this phase it conducts the one multi-threaded
scenario it was designed to conduct. After all participating threads have exited, either by
returning normally or throwing an exception, the conduct
method
will complete, either by returning normally or throwing an exception. As soon as
the conduct
method completes, the Conductor
enters its defunct phase. Once the Conductor
has conducted
a multi-threaded scenario, it is defunct and can't be reused. To run the same test again,
you'll need to create a new instance of Conductor
.
Here's an example of the use of Conductor
to test the ArrayBlockingQueue
class from java.util.concurrent
:
import org.scalatest.fixture.FunSuite import org.scalatest.matchers.ShouldMatchers import java.util.concurrent.ArrayBlockingQueue import org.scalatest.concurrent.Conductors class ArrayBlockingQueueSuite extends FunSuite with ShouldMatchers with Conductors { test("calling put on a full queue blocks the producer thread") { val conductor = new Conductor import conductor._ val buf = new ArrayBlockingQueue[Int](1) thread("producer") { buf put 42 buf put 17 beat should be (1) } thread("consumer") { waitForBeat(1) buf.take should be (42) buf.take should be (17) } whenFinished { buf should be ('empty) } } }
When the test shown is run, it will create one thread named producer and another named
consumer. The producer thread will eventually execute the code passed as a by-name
parameter to thread("producer")
:
buf put 42 buf put 17 beat should be (1)
Similarly, the consumer thread will eventually execute the code passed as a by-name parameter
to thread("consumer")
:
waitForBeat(1) buf.take should be (42) buf.take should be (17)
The thread
calls create the threads and starts them, but they will not immediately
execute the by-name parameter passed to them. They will first block, waiting for the Conductor
to give them a green light to proceed.
The next call in the test is whenFinished
. This method will first call conduct
on
the Conductor
, which will wait until all threads that were created (in this case, producer and consumer) are
at the "starting line", i.e., they have all started and are blocked, waiting on the green light.
The conduct
method will then give these threads the green light and they will
all start executing their blocks concurrently.
When the threads are given the green light, the beat is 0. The first thing the producer thread does is put 42 in
into the queue. As the queue is empty at this point, this succeeds. The producer thread next attempts to put a 17
into the queue, but because the queue has size 1, this can't succeed until the consumer thread has read the 42
from the queue. This hasn't happened yet, so producer blocks. Meanwhile, the consumer thread's first act is to
call waitForBeat(1)
. Because the beat starts out at 0, this call will block the consumer thread.
As a result, once the producer thread has executed buf put 17
and the consumer thread has executed
waitForBeat(1)
, both threads will be blocked.
The Conductor
maintains a clock that wakes up periodically and checks to see if all threads
participating in the multi-threaded scenario (in this case, producer and consumer) are blocked. If so, it
increments the beat. Thus sometime later the beat will be incremented, from 0 to 1. Because consumer was
waiting for beat 1, it will wake up (i.e., the waitForBeat(1)
call will return) and
execute the next line of code in its block, buf.take should be (42)
. This will succeed, because
the producer thread had previously (during beat 0) put 42 into the queue. This act will also make
producer runnable again, because it was blocked on the second put
, which was waiting for another
thread to read that 42.
Now both threads are unblocked and able to execute their next statement. The order is
non-deterministic, and can even be simultaneous if running on multiple cores. If the consumer
thread
happens to execute buf.take should be (17)
first, it will block (buf.take
will not return), because the queue is
at that point empty. At some point later, the producer thread will execute buf put 17
, which will
unblock the consumer thread. Again both threads will be runnable and the order non-deterministic and
possibly simulataneous. The producer thread may charge ahead and run its next statement, beat should be (1)
.
This will succeed because the beat is indeed 1 at this point. As this is the last statement in the producer's block,
the producer thread will exit normally (it won't throw an exception). At some point later the consumer thread will
be allowed to complete its last statement, the buf.take
call will return 17. The consumer thread will
execute 17 should be (17)
. This will succeed and as this was the last statement in its block, the consumer will return
normally.
If either the producer or consumer thread had completed abruptbly with an exception, the conduct
method
(which was called by whenFinished
) would have completed abruptly with an exception to indicate the test
failed. However, since both threads returned normally, conduct
will return. Because conduct
doesn't
throw an exception, whenFinished
will execute the block of code passed as a by-name parameter to it: buf should be ('empty)
.
This will succeed, because the queue is indeed empty at this point. The whenFinished
method will then return, and
because the whenFinished
call was the last statement in the test and it didn't throw an exception, the test completes successfully.
This test tests ArrayBlockingQueue
, to make sure it works as expected. If there were a bug in ArrayBlockingQueue
such as a put
called on a full queue didn't block, but instead overwrote the previous value, this test would detect
it. However, if there were a bug in ArrayBlockingQueue
such that a call to take
called on an empty queue
never blocked and always returned 0, this test might not detect it. The reason is that whether the consumer thread will ever call
take
on an empty queue during this test is non-deterministic. It depends on how the threads get scheduled during beat 1.
What is deterministic in this test, because the consumer thread blocks during beat 0, is that the producer thread will definitely
attempt to write to a full queue. To make sure the other scenario is tested, you'd need a different test:
test("calling take on an empty queue blocks the consumer thread") { val conductor = new Conductor import conductor._ val buf = new ArrayBlockingQueue[Int](1) thread("producer") { waitForBeat(1) buf put 42 buf put 17 } thread("consumer") { buf.take should be (42) buf.take should be (17) beat should be (1) } whenFinished { buf should be ('empty) } }
In this test, the producer thread will block, waiting for beat 1. The consumer thread will invoke buf.take
as its first act. This will block, because the queue is empty. Because both threads are blocked, the Conductor
will at some point later increment the beat to 1. This will awaken the producer thread. It will return from its
waitForBeat(1)
call and execute buf put 42
. This will unblock the consumer thread, which will
take the 42, and so on.
The problem that Conductor
is designed to address is the difficulty, caused by the non-deterministic nature
of thread scheduling, of testing classes, traits, and libraries that are intended to be used by multiple threads.
If you just create a test in which one thread reads from an ArrayBlockingQueue
and
another writes to it, you can't be sure that you have tested all possible interleavings of threads, no matter
how many times you run the test. The purpose of Conductor
is to enable you to write tests with deterministic interleavings of threads. If you write one test for each possible
interleaving of threads, then you can be sure you have all the scenarios tested. The two tests shown here, for example,
ensure that both the scenario in which a producer thread tries to write to a full queue and the scenario in which a
consumer thread tries to take from an empty queue are tested.
Class Conductor
was inspired by the
MultithreadedTC project,
created by Bill Pugh and Nat Ayewah of the University of Maryland.
Although useful, bear in mind that a Conductor
's results are not guaranteed to be
accurate 100% of the time. The reason is that it uses java.lang.Thread
's getState
method to
decide when to advance the beat. This use goes against the advice given in the Javadoc documentation for
getState
, which says, "This method is designed for use in monitoring of the system state, not for
synchronization." In short, sometimes the return value of getState
occasionally may be inacurrate,
which in turn means that sometimes a Conductor
could decide to advance the beat too early. In practice,
Conductor
has proven to be very helpful when developing thread safe classes. It is also useful in
for regression tests, but you may have to tolerate occasional false negatives.
Configuration object for asynchronous constructs, such as those provided by traits Eventually
and
AsyncAssertions
.
Configuration object for asynchronous constructs, such as those provided by traits Eventually
and
AsyncAssertions
.
The default values for the parameters are:
Configuration Parameter | Default Value |
---|---|
timeout
|
scaled(150 milliseconds)
|
interval
|
scaled(15 milliseconds)
|
the maximum amount of time to wait for an asynchronous operation to complete before giving up and throwing
TestFailedException
.
the amount of time to sleep between each check of the status of an asynchronous operation when polling
The total number of tests that are expected to run when this Suite
's run
method is invoked.
The total number of tests that are expected to run when this Suite
's run
method is invoked.
a Filter
with which to filter tests to count based on their tags
An immutable IndexedSeq
of this SuiteMixin
object's nested Suite
s.
An immutable IndexedSeq
of this SuiteMixin
object's nested Suite
s. If this SuiteMixin
contains no nested Suite
s,
this method returns an empty IndexedSeq
.
The fully qualified name of the class that can be used to rerun this suite.
The fully qualified name of the class that can be used to rerun this suite.
Runs this suite of tests.
Runs this suite of tests.
an optional name of one test to execute. If None
, all relevant tests should be executed.
I.e., None
acts like a wildcard that means execute all relevant tests in this Suite
.
the Args
for this run
a Status
object that indicates when all tests and nested suites started by this method have completed, and whether or not a failure occurred.
NullArgumentException
if any passed parameter is null
.
Runs zero to many of this suite's nested suites.
Runs zero to many of this suite's nested suites.
the Args
for this run
a Status
object that indicates when all nested suites started by this method have completed, and whether or not a failure occurred.
NullArgumentException
if args
is null
.
Runs a test.
Runs a test.
the name of one test to execute.
the Args
for this run
a Status
object that indicates when the test started by this method has completed, and whether or not it failed .
NullArgumentException
if any of testName
or args
is null
.
Runs zero to many of this suite's tests.
Runs zero to many of this suite's tests.
an optional name of one test to run. If None
, all relevant tests should be run.
I.e., None
acts like a wildcard that means run all relevant tests in this Suite
.
the Args
for this run
a Status
object that indicates when all tests started by this method have completed, and whether or not a failure occurred.
NullArgumentException
if either testName
or args
is null
.
This suite's style name.
This suite's style name.
This lifecycle method provides a string that is used to determine whether this suite object's style is one of the chosen styles for the project.
A string ID for this Suite
that is intended to be unique among all suites reported during a run.
A string ID for this Suite
that is intended to be unique among all suites reported during a run.
The suite ID is intended to be unique, because ScalaTest does not enforce that it is unique. If it is not unique, then you may not be able to uniquely identify a particular test of a particular suite. This ability is used, for example, to dynamically tag tests as having failed in the previous run when rerunning only failed tests.
this Suite
object's ID.
A user-friendly suite name for this Suite
.
A user-friendly suite name for this Suite
.
This trait's
implementation of this method returns the simple name of this object's class. This
trait's implementation of runNestedSuites
calls this method to obtain a
name for Report
s to pass to the suiteStarting
, suiteCompleted
,
and suiteAborted
methods of the Reporter
.
this Suite
object's suite name.
A Map
whose keys are String
names of tagged tests and
whose associated values are the Set
of tag names for the test.
A Map
whose keys are String
names of tagged tests and
whose associated values are the Set
of tag names for the test. If a test has no associated tags, its name
does not appear as a key in the returned Map
. If this Suite
contains no tests with tags, this
method returns an empty Map
.
Subclasses may override this method to define and/or discover tags in a custom manner, but overriding method implementations
should never return an empty Set
as a value. If a test has no tags, its name should not appear as a key in the
returned Map
.
Provides a TestData
instance for the passed test name, given the passed config map.
Provides a TestData
instance for the passed test name, given the passed config map.
This method is used to obtain a TestData
instance to pass to withFixture(NoArgTest)
and withFixture(OneArgTest)
and the beforeEach
and afterEach
methods
of trait BeforeAndAfterEach
.
the name of the test for which to return a TestData
instance
the config map to include in the returned TestData
a TestData
instance for the specified test, which includes the specified config map
A Set
of test names.
A Set
of test names. If this Suite
contains no tests, this method returns an empty Set
.
Although subclass and subtrait implementations of this method may return a Set
whose iterator produces String
test names in a well-defined order, the contract of this method does not required a defined order. Subclasses are free to
implement this method and return test names in either a defined or undefined order.
Gets the current value of the clock.
Gets the current value of the clock. Primarily useful in assert statements.
the current tick value
Returns an Interval
configuration parameter containing the passed value, which
specifies the amount of time to sleep after a retry.
Returns an Interval
configuration parameter containing the passed value, which
specifies the amount of time to sleep after a retry.
Check if the clock has been frozen by any threads.
Check if the clock has been frozen by any threads. (The only way a thread can freeze the clock is by calling withClockFrozen.)
Implicit PatienceConfig
value providing default configuration values.
Implicit PatienceConfig
value providing default configuration values.
To change the default configuration, override or hide this def
with another implicit
PatienceConfig
containing your desired default configuration values.
Scales the passed Span
by the Double
factor returned
by spanScaleFactor
.
Scales the passed Span
by the Double
factor returned
by spanScaleFactor
.
The Span
is scaled by invoking its scaledBy
method,
thus this method has the same behavior:
The value returned by spanScaleFactor
can be any positive number or zero,
including a fractional number. A number greater than one will scale the Span
up to a larger value. A fractional number will scale it down to a smaller value. A
factor of 1.0 will cause the exact same Span
to be returned. A
factor of zero will cause Span.ZeroLength
to be returned.
If overflow occurs, Span.Max
will be returned. If underflow occurs,
Span.ZeroLength
will be returned.
IllegalArgumentException
if the value returned from spanScaleFactor
is less than zero
The factor by which the scaled
method will scale Span
s.
The factor by which the scaled
method will scale Span
s.
The default implementation of this method will return the span scale factor that
was specified for the run, or 1.0 if no factor was specified. For example, you can specify a span scale factor when invoking ScalaTest
via the command line by passing a -F
argument to Runner
.
Create a new thread that will execute the given function.
Create a new thread that will execute the given function. If the test is started, then the thread will run the function immediately. If it is not yet started, the Thread will wait to run the function until all threads are up and ready to go.
the function to be executed by the thread
Create a new thread that will execute the given function.
Create a new thread that will execute the given function. If the test is started, then the thread will run the function immediately. If it is not yet started, the Thread will wait to run the function until all threads are up and ready to go.
the name of the thread
the function to be executed by the thread
Returns a Timeout
configuration parameter containing the passed value, which
specifies the maximum amount to wait for an asynchronous operation to complete.
Returns a Timeout
configuration parameter containing the passed value, which
specifies the maximum amount to wait for an asynchronous operation to complete.
Force the current thread to block until the thread clock reaches the specified value, at which point the current thread is unblocked.
Force the current thread to block until the thread clock reaches the specified value, at which point the current thread is unblocked.
Register a function to be executed after the simulation has finished.
Register a function to be executed after the simulation has finished.
Run the passed function, ensuring the clock does not advance while the function is running (has not yet returned or thrown an exception).
Run the passed function, ensuring the clock does not advance while the function is running (has not yet returned or thrown an exception).
Creates and initializes a private instance variable with a new Conductor,
ensuring it is visible to any thread, invokes the passed test function,
and invokes conduct
on the Conductor
, if it
was not already invoked by the test.
Creates and initializes a private instance variable with a new Conductor,
ensuring it is visible to any thread, invokes the passed test function,
and invokes conduct
on the Conductor
, if it
was not already invoked by the test.
This trait is stackable with other traits that override withFixture(NoArgTest)
, because
instead of invoking the test function directly, it delegates responsibility for invoking the test
function to super.withFixture(NoArgTest)
.
the no-arg test function to run with a fixture
Trait that provides each test with access to a new
Conductor
via methods.Here's an example of the use of this trait to test the
ArrayBlockingQueue
concurrency abstraction fromjava.util.concurrent
:For an explanation of how these tests work, see the documentation for
Conductors
.