Trait/Object

org.scalatest

Retries

Related Docs: object Retries | package scalatest

Permalink

trait Retries extends AnyRef

Provides methods that can be used in withFixture implementations to retry tests in various scenarios.

Trait Retries is intended to help you deal with “flickers”—tests that usually pass, but occasionally fail. The best way to deal with such tests is to fix them so they always pass. Sometimes, however, this is not practical. In such cases, flickers can waste your time by forcing you to investigate test failures that turn out to be flickers. Or worse, like the boy who cried wolf, the flickers may train you an your colleagues to not pay attention to failures such that you don't notice real problems, at least not in a timely manner.

Trait Retries offers methods that will retry a failed and/or canceled test once, on the same thread, with or without a delay. These methods take a block that results in Outcome, and are intended to be used in withFixture methods. You should be very selective about which tests you retry, retrying those for which you have good evidence to conclude they are flickers. Thus it is recommended you only retry tests that are tagged with Retryable, and only tag tests as such once they have flickered consistently for a while, and only after you invested a reasonable effort into fixing them properly.

Here's an example showing how you might use Retries:

package org.scalatest.examples.tagobjects.retryable

import org.scalatest._
import tagobjects.Retryable

class SetSpec extends FlatSpec with Retries {

  override def withFixture(test: NoArgTest) = {
    if (isRetryable(test))
      withRetry { super.withFixture(test) }
    else
      super.withFixture(test)
  }

  "An empty Set" should "have size 0" taggedAs(Retryable) in {
    assert(Set.empty.size === 0)
  }
}

Source
Retries.scala
Linear Supertypes
AnyRef, Any
Known Subclasses
Retries
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Retries
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  6. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  7. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  9. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  10. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  11. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  12. def isRetryable(testData: TestData): Boolean

    Permalink

    Indicates whether the test described by the given TestData includes the tag org.scalatest.tags.Retryable.

    Indicates whether the test described by the given TestData includes the tag org.scalatest.tags.Retryable.

    This method provides an easy way to selectively retry just tests that are flickering. You can annotated such problematic tests with Retryable, and just retry those. Here's what it might look like:

    override def withFixture(test: NoArgTest) = {
  if (isRetryable(test))
    withRetry { super.withFixture(test) }
  else
    super.withFixture(test)
}

  13. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  14. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  15. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  16. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  17. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any

  18. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  19. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  20. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  21. def withRetry(delay: Span)(blk: ⇒ Outcome): Outcome

    Permalink

    Retries the given block with a given delay if the Outcome of executing the block is either Failed or Canceled.

    Retries the given block with a given delay if the Outcome of executing the block is either Failed or Canceled.

    The behavior of this method is defined in the table below. The first two rows show the main "retry" behavior: if executing the block initially fails, and on retry it succeeds, the result is Canceled. The purpose of this is to deal with "flickering" tests by downgrading a failure that succeeds on retry to a cancelation. Or, if executing the block initially results in Canceled, and on retry it succeeds, the result is Succeeded. The purpose of this is to deal with tests that intermittently cancel by ignoring a cancelation that succeeds on retry.

    In the table below, if the “Retry Outcome” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, after sleeping the given delay.

    First OutcomeRetry OutcomeResult
    Failed Succeeded Canceled (the Succeeded and Failed are discarded; the exception from the Failed is the cause of the exception in the Canceled)
    Canceled Succeeded Succeeded (the Canceled is discarded)
    Succeeded — Succeeded (no retry)
    Pending — Pending (no retry)
    Failed Failed the first Failed (the second Failed is discarded)
    Failed Pending the Failed (the Pending is discarded)
    Failed Canceled the Failed (the Canceled is discarded)
    Canceled Canceled the first Canceled (the second Canceled is discarded)
    Canceled Pending the Canceled (the Pending is discarded)
    Canceled Failed the Failed (the Canceled is discarded)

    delay

    the amount of time to sleep before retrying

    blk

    the block to execute and potentially retry

  22. def withRetry(blk: ⇒ Outcome): Outcome

    Permalink

    Retries the given block immediately (with no delay) if the Outcome of executing the block is either Failed or Canceled.

    Retries the given block immediately (with no delay) if the Outcome of executing the block is either Failed or Canceled.

    The behavior of this method is defined in the table below. The first two rows show the main "retry" behavior: if executing the block initially fails, and on retry it succeeds, the result is Canceled. The purpose of this is to deal with "flickering" tests by downgrading a failure that succeeds on retry to a cancelation. Or, if executing the block initially results in Canceled, and on retry it succeeds, the result is Succeeded. The purpose of this is to deal with tests that intermittently cancel by ignoring a cancelation that succeeds on retry.

    In the table below, if the “Retry Outcome” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, with no delay.

    First OutcomeRetry OutcomeResult
    Failed Succeeded Canceled (the Succeeded and Failed are discarded; the exception from the Failed is the cause of the exception in the Canceled)
    Canceled Succeeded Succeeded (the Canceled is discarded)
    Succeeded — Succeeded (no retry)
    Pending — Pending (no retry)
    Failed Failed the first Failed (the second Failed is discarded)
    Failed Pending the Failed (the Pending is discarded)
    Failed Canceled the Failed (the Canceled is discarded)
    Canceled Canceled the first Canceled (the second Canceled is discarded)
    Canceled Pending the Canceled (the Pending is discarded)
    Canceled Failed the Failed (the Canceled is discarded)

    blk

    the block to execute and potentially retry

  23. def withRetryOnCancel(delay: Span)(blk: ⇒ Outcome): Outcome

    Permalink

    Retries the given block after the given delay if the Outcome of executing the block is Canceled.

    Retries the given block after the given delay if the Outcome of executing the block is Canceled.

    The behavior of this method is defined in the table below. The first row shows the main "retry" behavior: if executing the block initially results in Canceled, and on retry it succeeds, the result is Succeeded. The purpose of this is to deal with tests that intermittently cancel by ignoring a cancelation that succeeds on retry.

    In the table below, if the “Retry Outcome” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, after the given delay.

    First OutcomeRetry OutcomeResult
    Canceled Succeeded the Succeeded (the Canceled is discarded)
    Succeeded — Succeeded (no retry)
    Pending — Pending (no retry)
    Failed — Failed (no retry)
    Canceled Canceled the first Canceled (the second Canceled is discarded)
    Canceled Pending the Canceled (the Pending is discarded)
    Canceled Failed the Failed (the Canceled is discarded)

    delay

    the amount of time to sleep before retrying

    blk

    the block to execute and potentially retry

  24. def withRetryOnCancel(blk: ⇒ Outcome): Outcome

    Permalink

    Retries the given block immediately (with no delay) if the Outcome of executing the block is Canceled.

    Retries the given block immediately (with no delay) if the Outcome of executing the block is Canceled.

    The behavior of this method is defined in the table below. The first row shows the main "retry" behavior: if executing the block initially results in Canceled, and on retry it succeeds, the result is Succeeded. The purpose of this is to deal with tests that intermittently cancel by ignoring a cancelation that succeeds on retry.

    In the table below, if the “Retry Outcome” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, with no delay.

    First OutcomeRetry OutcomeResult
    Canceled Succeeded the Succeeded (the Canceled is discarded)
    Succeeded — Succeeded (no retry)
    Pending — Pending (no retry)
    Failed — Failed (no retry)
    Canceled Canceled the first Canceled (the second Canceled is discarded)
    Canceled Pending the Canceled (the Pending is discarded)
    Canceled Failed the Failed (the Canceled is discarded)

    blk

    the block to execute and potentially retry

  25. def withRetryOnFailure(delay: Span)(blk: ⇒ Outcome): Outcome

    Permalink

    Retries the given block immediately with the given delay if the Outcome of executing the block is Failed.

    Retries the given block immediately with the given delay if the Outcome of executing the block is Failed.

    The behavior of this method is defined in the table below. The first row shows the main "retry" behavior: if executing the block initially fails, and on retry it succeeds, the result is Canceled. The purpose of this is to deal with "flickering" tests by downgrading a failure that succeeds on retry to a cancelation.

    In the table below, if the “Retry Outcome” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, after the given delay.

    First OutcomeRetry OutcomeResult
    Failed Succeeded Canceled (the Succeeded and Failed are discarded; the exception from the Failed is the cause of the exception in the Canceled)
    Succeeded — Succeeded (no retry)
    Pending — Pending (no retry)
    Canceled — the Canceled (no retry)
    Failed Failed the first Failed (the second Failed is discarded)
    Failed Pending the Failed (the Pending is discarded)
    Failed Canceled the Failed (the Canceled is discarded)

    delay

    the amount of time to sleep before retrying

    blk

    the block to execute and potentially retry

  26. def withRetryOnFailure(blk: ⇒ Outcome): Outcome

    Permalink

    Retries the given block immediately (with no delay) if the Outcome of executing the block is Failed.

    Retries the given block immediately (with no delay) if the Outcome of executing the block is Failed.

    The behavior of this method is defined in the table below. The first row shows the main "retry" behavior: if executing the block initially fails, and on retry it succeeds, the result is Canceled. The purpose of this is to deal with "flickering" tests by downgrading a failure that succeeds on retry to a cancelation.

    In the table below, if the “Retry Outcome” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, with no delay.

    First OutcomeRetry OutcomeResult
    Failed Succeeded Canceled (the Succeeded and Failed are discarded; the exception from the Failed is the cause of the exception in the Canceled)
    Succeeded — Succeeded (no retry)
    Pending — Pending (no retry)
    Canceled — the Canceled (no retry)
    Failed Failed the first Failed (the second Failed is discarded)
    Failed Pending the Failed (the Pending is discarded)
    Failed Canceled the Failed (the Canceled is discarded)

    blk

    the block to execute and potentially retry

Inherited from AnyRef

Inherited from Any

Ungrouped