Combo-Test is a command-line tool that generates test suites with a minimal set of test cases controllable by the System Under Test inputs (components and their values) and a pre-configured strength control (the number of combinations of components that are to be exercised). The tool was developed at the Australian e-Health research Centre (AEHRC) using Java and utilises a JDBC connection to a pre-installed database.
The efficiency of a test suite can be measured by the number of test cases necessary to uncover the defects in a system (providing the time to execute a test is approximately constant for all tests in that suite). All permutations and combinations of inputs must be exercised to satisfy exhaustive testing which is the gold standard in black-box testing and (providing failures are discrete and identifiable) can be regarded as the method by which the greatest number of functional defects in the system will be detected.
The other end of the test suite scale is single mode testing where the tests exercise each single component while disregarding the effects of combinations of inputs. This considers a component as a singleton and is normally executed as part of unit testing.
Combo-Test uses a thresholding technique to work through an exhaustive table finding matches against an array of n-sets. Several local minima are discovered for each execution of the tool, the one with the least number of tests is the one chosen.
Experimental results on a range of test projects assisted with the Combo-Test tool at the AEHRC shows that executing test suites generated by the Combo-Test tool provided a comparable defect hit rate as running exhaustive suites, but with a fraction of the execution effort.
Combo-Test is currently available as an open sourced application.
Last Updated on Thursday, 29 September 2011 12:42