Priority Based System Test Method
Original Publication Date: 2001-Sep-01
Included in the Prior Art Database: 2003-Jun-20
Disclosed is a method for defining a priority-based representation of all possible system test cases. To track test coverage the method incorporates a set of dependency rules that help deterministically restart the correct subset of dependent tests in the event of a test case failure. The method disclosed facilitates introducing clever test case reduction algorithms to increase the probability of finding bugs without executing every possible test combination. This method is particularly useful for implementing an automated test harness. The method makes use of an n- dimensional matrix to represent all possible combinations of the system components. Each of the n components are represented by an axis. Each cell of the matrix is identified by its position with respect to the n axes and represents a unique test case. The test case combination is determined based on the values of each of the axes. For the 3-axes example shown in Figure (i), cell A (x 3 y 3 z 1 represents the test case for a combination of Application 3, Server node 3 and Operating System 1. Figure (i): 3-dimensional matrix test case representation. To start with all the cells of the test status matrix are set to a ToBeTested status. Next, the cells that represent non-supported component combinations are marked with a NotSupported status. Example: cell B (x 3 y 4 z 1 marked in black color. The component combinations represented by these cells are never tested. The remainder of the matrix represent supported component combinations that need to be tested. For each test that executes correctly and passes the test criteria, the corresponding cell is marked with a Success status. Example: the green colored cell A (x 3 y 3 z 1 ). The testing may be performed in any order. When a test fails the corresponding cell is marked with a Failure status. Example: the red colored cell C (x 7 y 3 z 1 ).