Adding support for new testing and build frameworks

Adding support for a new frameworks

To load-balance tests across multiple processes, TLB needs to hook up with 2 kinds of frameworks.

1. Testing framework: This is the framework that actually runs the tests (for instance junit or rspec etc)
2. Build framework: This is the framework that calls out to the tests (command line invocation, Ant, Rake etc)

Adding a new Testing framework

TLB assumes that a test framework provides an option to specify a list of file resources that get executed. The list is passed to the criteria chain. Splitter criterion prunes the file resource list. After pruning, the list of the file resources is passed through the orderer, where it gets re-ordered. The contract is that the orderer does not change the number of file resources.

The final list of file resources is what is fed into the test framework for execution.

Once the tests are executed, TLB needs a way to capture the test result and the time each test took to execute in order to report back to the TLB Server. This is the feedback that is used for partitioning tests in future. This feedback is what allows TLB to partion and order tests accurately and sensibly.

Build tool

Supporting build tools is a matter of implementing the end user interface which delegate to the Splitter and Orderer. Build tool integration goes a little further and does the plumbing work of attaching listeners to the test running framework so the feedback can be posted. For instance, Ant support hooks up at the fileset-filter level for partitioning and re-ordering and attaches a JUnitResultFormatter with the test task, so the feedback gets posted.