Browse Prior Art Database

Statistical Method to Optimize Regression Test Time

IP.com Disclosure Number: IPCOM000247518D
Publication Date: 2016-Sep-14
Document File: 5 page(s) / 112K

Publishing Venue

The IP.com Prior Art Database

Abstract

Regression testing is an expensive testing process used to re-validate software as it evolves. How to conduct an efficient regression testing is very important on the aspects of effectiveness and cost. The goal is to achieve better effectiveness with lower cost. One typical method is retest-all methodology which re-uses all previously developed test cases, executing them on the modified program. This costs too much time and is not possible when the schedule is tight. Sometimes fixing a defect in a software component causes malfunctioning in another component, so effort is taken on how to select the re-used test cases for regression bucket. It is necessary to identify a subset of significant test cases that with high probability ensure a sufficient and exhaustive test in the minimum possible time in order to avoid regressions, while maintaining affordable test costs. In this article, a statistical method to optimize regression test time is proposed and it is based on the feature of only executing test cases related to those components having a percentage of impact exceeding a threshold due to their code changes on other software components. A historical database storing the association among all the components and the fixes applied to them is constantly updated and used as a base for calculating the new values of the dependency factor each time a new incoming defect is raised.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 52% of the total text.

Page 01 of 5

Statistical Method to Optimize Regression Test Time

    When one has to make a fix to resolve a defect in a software program, there is usually not enough time to perform a complete suite of test cases to avoid regressions which are expensive testing methods. Therefore, sometimes, fixing a defect in a software component causes malfunctioning in another component. It is necessary to identify a subset of significant test cases that with high probability ensure a sufficient and exhaustive test in the minimum possible time in order to avoid regressions, while maintaining affordable test costs.

    Assume that you have a history of information about software defects, the related fixes and subsequent defects (regressions) arising as a result of changes or fixes made to the various components. You can build a diagram showing, for each component of the software, the percentage (or the weight) that a change to a component impacts on all the other components. The more the information is accurate and historical, the more the diagram is accurate in reflecting the impacts of changes among software components. Considering that each software component has its associated set of test cases, you have a statistical order for prioritizing the execution of the test cases, which in turn ensures the optimization of regression test time.

    Some existing solutions for optimizing the test time in terms of test cases selection are based on static and dynamic analysis of the code. Having a statistical way to select test cases based on historical information gives you a more precise measure of why changes to the some component of a software have impacted other components. This is true especially for dated software products used by many customers on the field.

    In the diagram below are represented the various component of a software. Each line in the diagram has two percentages (or weights), one for each direction. The percentage indicates the historical probability that a change in the code of a component had impacts on the other components. For example, a change in the component 6 had 98% of historical probability to affect the component 1, but a change in the component 1 had 23% of historical probability to affect the component 6. Based on time and effort that you can put in the test phase, you can start executing test cases from those related to the components with the highest percentage of impact, then proceeding with the test cases related to the components with the lowest percentage of impact. Depending on your resources constraints, you can decide a safety threshold and execute only test cases related to components with a percen...