Browse Prior Art Database

Mechanism And Algorithm For Optimizing Test Case Execution Using Historical Data

IP.com Disclosure Number: IPCOM000198310D
Publication Date: 2010-Aug-04
Document File: 5 page(s) / 28K

Publishing Venue

The IP.com Prior Art Database

Abstract

The disclosed describes a mechanism for optimally prioritizing test cases for execution. The mechanism collects and stores statistics from the execution of test cases and employs a ranking model using a specific algorithm. test case rank indicates priority level.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 36% of the total text.

Page 1 of 5

Mechanism And Algorithm For Optimizing Test Case Execution Using Historical

Data

A test case validates a specific feature or set of features and is typically composed of

which are the most granular level of testing.

The development of large software products involves the creation of a set of test cases.

The set of test cases associated with a product grows as the product evolves and

engineers incorporate new features. Complex and mature software products such as

IBM DB2TM often require hundreds of thousands of test cases for validation.

Managing, executing and maintaining a large set of test cases presents a number of

challenges, such as:
1. Prioritizing the execution of test cases.
2. Selecting optimal sets to execute with limited resources (i.e., time, machine).
3. Identifying test cases which require significant effort to maintain. (Engineers use

such sets to focus refactoring and reengineering efforts). Long-running test

cases without a history of identifying quality regressions are candidates for

deprecation.
4. Efficiently distributing work to a set of machines for execution.
5. Efficiently tolerating non-robust test cases until they can be reengineered or

deprecated.

Current solutions present methods that are based on code coverage of test cases. The

disclosed algorithm and method are unique because they are based on historical data

from existing test cases independent of coverage.

This invention uses data from the Test Case Aggregate Results and Classification

Model (TARCM) to prioritize the order in which test cases are dispatched for the

purposes of gaining an initial indication of product quality and ensuring the results of

historically problematic test cases are available to testers first. This optimally prioritizes

test cases for execution.

Another application of the model involves using it to classify

test cases and to optimize test case execution based on a given criteria. The key

attributes include ordering of test case execution so that:

[circle6]The process assigns test cases a ranking from a number of criteria. For

example, a test case is valued more favorably if it has a history of discovering

code regressions,

whereas a test case is valued less favorably if it requires a

long time to execute. The mechanism executes the most valuable test cases

first.

one or more test units,

1

Page 2 of 5

[circle6]The process is adaptive and automatic. If the characteristics of one of the test

cases or the product change, then the ranking system adjusts and re-ranks the

test case.

[circle6]Developers can use the ranking and classification system to focus

reengineering efforts or identify candidates for deprecation.

The invention uses TARCM to assign a test case rank based on its execution duration

and its history of finding code defects.

Algorithm:

1. Rank each test case using data from the TARCM. Rank = (

A

A and B constants can be used to control the importance of number of code

regressions found and the average runtime of the te...