edu.umd.cs.mtc
Class TestFramework

java.lang.Object
  extended by edu.umd.cs.mtc.TestFramework

public class TestFramework
extends Object

This class provides static methods to perform a MultithreadedTestCase. The method runOnce(MultithreadedTestCase) can be used to run a MultithreadedTestCase once. The method runManyTimes(MultithreadedTestCase, int) can be used to run a MultithreadedTestCase multiple times (to see if different interleavings produce different behaviors).

Each test case starts by running the initialize method, followed by all the thread methods in different threads, and finally the finish method when all threads have finished. The thread methods are run in a new thread group, and are regulated by a separate clock thread. The clock thread checks periodically to see if all threads are blocked. If all threads are blocked and at least one is waiting for a tick, the clock thread advances the clock to the next desired tick. (A slight delay -- about a clock period -- is applied before advancing the clock to ensure that this is not done prematurely and any threads trying to unblock are given a chance to do so.)

The clock thread also detects deadlock (when all threads are blocked, none are waiting for a tick, and none are in state TIMED_WAITING), and can stop a test that is going on too long (a thread is in state RUNNABLE for too long.)

Since the test case threads are placed in a new thread group, any other threads created by these test cases will be placed in this thread group by default. All threads in the thread group will be considered by the clock thread when deciding whether to advance the clock, declare a deadlock, or stop a long-running test.

The framework catches exceptions thrown in the threads and propagates them to the JUnit test (It also throws AssertionErrors)

This class also defines a number of parameters to be used to control the tests. Set command line parameter -Dtunit.runLimit=n to cause a test case to fail if at least one thread stays in a runnable state for more than n seconds without becoming blocked or waiting for a metronome tick. Set command line parameter -Dtunit.clockPeriod=p to cause the clock thread to check the status of all the threads every p milliseconds.

Since:
1.0
Author:
William Pugh, Nathaniel Ayewah
See Also:
MultithreadedTestCase, runOnce(MultithreadedTestCase), runManyTimes(MultithreadedTestCase, int)

Field Summary
static String CLOCKPERIOD_KEY
          Command line key for indicating the regularity (in milliseconds) with which the clock thread regulates the thread methods.
static Integer DEFAULT_CLOCKPERIOD
          The default clock period in milliseconds
static Integer DEFAULT_RUNLIMIT
          The default run limit in seconds
static String RUNLIMIT_KEY
          Command line key for indicating the time limit (in seconds) for runnable threads.
 
Constructor Summary
TestFramework()
           
 
Method Summary
static void addSetUpAndTearDown(MultithreadedTest mtc, TestCase tc)
          Update a given test to call "setUp" and "tearDown" before and after running the test respectively.
static TestSuite buildTestSuite(Class<?> c)
          Scan through a given class c to find any inner classes that implement Test.
static TestSuite buildTestSuite(Class<?> c, String suiteName)
          Scan through a given class c to find any inner classes that implement Test.
static void runInstrumentedManyTimes(MultithreadedTestCase test, int count, int[] failureCount)
          Run multithreaded test case multiple times using the default or global settings for clock period and run limit.
static void runManyTimes(MultithreadedTestCase test, int count)
          Run multithreaded test case multiple times using the default or global settings for clock period and run limit.
static void runManyTimes(MultithreadedTestCase test, int count, Integer clockPeriod, Integer runLimit)
          Run multithreaded test case multiple times.
static void runOnce(MultithreadedTestCase test)
          Run a multithreaded test case once, using the default or global settings for clock period and run limit
static void runOnce(MultithreadedTestCase test, Integer clockPeriod, Integer runLimit)
          Run multithreaded test case once.
static void setGlobalClockPeriod(Integer v)
          Change/set the system property for the clock period
static void setGlobalRunLimit(Integer v)
          Change/set the system property for the run limit
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

CLOCKPERIOD_KEY

public static final String CLOCKPERIOD_KEY
Command line key for indicating the regularity (in milliseconds) with which the clock thread regulates the thread methods.

See Also:
Constant Field Values

RUNLIMIT_KEY

public static final String RUNLIMIT_KEY
Command line key for indicating the time limit (in seconds) for runnable threads.

See Also:
Constant Field Values

DEFAULT_CLOCKPERIOD

public static final Integer DEFAULT_CLOCKPERIOD
The default clock period in milliseconds


DEFAULT_RUNLIMIT

public static final Integer DEFAULT_RUNLIMIT
The default run limit in seconds

Constructor Detail

TestFramework

public TestFramework()
Method Detail

setGlobalClockPeriod

public static void setGlobalClockPeriod(Integer v)
Change/set the system property for the clock period

Parameters:
v - the new value for the clock period

setGlobalRunLimit

public static void setGlobalRunLimit(Integer v)
Change/set the system property for the run limit

Parameters:
v - the new value for the run limit

runInstrumentedManyTimes

public static void runInstrumentedManyTimes(MultithreadedTestCase test,
                                            int count,
                                            int[] failureCount)
                                     throws Throwable
Run multithreaded test case multiple times using the default or global settings for clock period and run limit. This method adds instrumentation to count the number of times failures occur (an exception is thrown). If the array failureCount is initialized to be of at least size 1, it returns this count in failureCount[0]. If failures do occur, it saves the first failure, and then throws it after running the test count times.

Parameters:
test - The multithreaded test case to run
count - the number of times to run the test case
failureCount - if this array is initialzed to at least size 1, the number of failures is returned in failureCount[0]
Throws:
Throwable - if there is at least one failure -- the first failure is thrown

runManyTimes

public static void runManyTimes(MultithreadedTestCase test,
                                int count)
                         throws Throwable
Run multithreaded test case multiple times using the default or global settings for clock period and run limit. The value of this is limited, since even running a test case a thousand or a million times may not expose any bugs dependent upon particular thread interleavings.

Parameters:
test - The multithreaded test case to run
count - the number of times to run the test case
Throws:
Throwable - -- if any of the test runs fails, the exception is thrown immediately without completing the rest of the test runs.

runManyTimes

public static void runManyTimes(MultithreadedTestCase test,
                                int count,
                                Integer clockPeriod,
                                Integer runLimit)
                         throws Throwable
Run multithreaded test case multiple times. The value of this is limited, since even running a test case a thousand or a million times may not expose any bugs dependent upon particular thread interleavings.

Parameters:
test - The multithreaded test case to run
count - the number of times to run the test case
clockPeriod - The period (in ms) between checks for the clock (or null for default or global setting)
runLimit - The limit to run the test in seconds (or null for default or global setting)
Throws:
Throwable - -- if any of the test runs fails, the exception is thrown immediately without completing the rest of the test runs.

runOnce

public static void runOnce(MultithreadedTestCase test)
                    throws Throwable
Run a multithreaded test case once, using the default or global settings for clock period and run limit

Parameters:
test - The multithreaded test case to run
Throws:
Throwable - if the test runs fails or causes an exception

runOnce

public static void runOnce(MultithreadedTestCase test,
                           Integer clockPeriod,
                           Integer runLimit)
                    throws Throwable
Run multithreaded test case once.

Parameters:
test - The multithreaded test case to run
clockPeriod - The period (in ms) between checks for the clock (or null for default or global setting)
runLimit - The limit to run the test in seconds (or null for default or global setting)
Throws:
Throwable - if the test runs fails or causes an exception

buildTestSuite

public static TestSuite buildTestSuite(Class<?> c)
Scan through a given class c to find any inner classes that implement Test. If the classes have a no-arg constructor, they are instantiated added them to a TestSuite. If the inner classes are not declared static then an instance of the class represented by c (created with a no-arg constructor) is used to construct the inner class. If no relevant inner classes are found, then an empty TestSuite is returned.

Parameters:
c - the class to scan for relevant inner classes
Returns:
A TestSuite containing one test for each relevant inner class

buildTestSuite

public static TestSuite buildTestSuite(Class<?> c,
                                       String suiteName)
Scan through a given class c to find any inner classes that implement Test. If the classes have a no-arg constructor, they are instantiated added them to a TestSuite. If the inner classes are not declared static then an instance of the class represented by c (created with a no-arg constructor) is used to construct the inner class. If no relevant inner classes are found, then an empty TestSuite is returned.

If the class is a TestCase, then an instance of it is passed to any non-static innerclass and the appropriate setUp and tearDown methods are called.

Parameters:
c - the class to scan for relevant inner classes
suiteName - A name for the TestSuite
Returns:
A TestSuite containing one test for each relevant inner class

addSetUpAndTearDown

public static void addSetUpAndTearDown(MultithreadedTest mtc,
                                       TestCase tc)
                                throws SecurityException,
                                       NoSuchMethodException
Update a given test to call "setUp" and "tearDown" before and after running the test respectively. Update is done by creating a new test. This assumes that the provided TestCase instance was used to create the Test, otherwise it does little good to call the "setUp" and "tearDown" method in the TestCase.

Parameters:
mtc - the test to update
tc - the TestCase that contains the setUp and tearDown methods called
Throws:
Any - exceptions that occur along the process. In this case, just use the old uninstrumented Test.
SecurityException
NoSuchMethodException