A method of dynamic value-ranking based regression test bucket selection
Publication Date: 2015-Jan-08
The IP.com Prior Art Database
With software product deliver more fast, we do not have enough time to run all test buckets for each product build.For guarantee the quality of each build, we need to do test bucket selection and just run the needed test buckets.How to balance the test time effort and product quality? Currently some existing test bucket selection is like, detect code change and count the related test cases into regression test bucket. But this kind of method will missing the impact result from the change delivered in historical product build. We provide a solution which can dynamically ranking the value of test case for each product build, then base on the test estimated completion time select the test case from high value to low value. For ranking the test case, we provide a formula which can base on two relationships which is relationship between test case and work item, work item in current build and work item in previous build, with these two relationships and test execution effort we can give each test case a value for each product build, then ranking is generated.
Page 01 of 3
A method of dynamic value
A method of dynamic value-
--ranking based regression test bucket selection
ranking based regression test bucket selection
Consider what change delivered in previous history build will relate to the change in current build and count the related test case of history change into current regression bucket.
In below diagram, you will see "Work item refer to the any record we use to track change request which includes: Defect or Task or Plan Item"
Key point1: History Track
In creating regression test bucket for Build v3, existing solution just consider the change in Build v3. But work item 5
which included in Build v3 has a related work item -- work item 3, so the test case 2 will also need to be take into consideration.
So regression buckets for Build v3 should consider test case 1, test case 2 and test case 3.
Key point2: Dynamic Ranking
A ranking mechanism will help to select the most needed test case for execution. And ranking is dynamic by each product build.
Before each test execution of each build, choose high value test cases to run and make sure that the sum of testing effort
Page 02 of 3
will not exceed the product plan execution time.In below circumstances,test case value be added.
Note: In condition 2, the test case related to the work item in previous build will also has value added.
Tracking work items which delivered in current build will help collect some test cases and tracking related work items which delivered in historical build will help collect some test cases related to those work items.So base on above method, test case value is different from each build, so the value ranking of test case is dynamic by build.
For calculating the value of a test case, we provide a formula which can help us.
Value of test case = (∑(R*S))/T
Note: value of test case is different from build to build.
Explanation of formula element:
R: relation rate between test case and work item
If test case related to the work item in current build, R is highest value. eg,10
If test case related to the work item delivered in previous build, R is 9
If test case related to the work item delivered in previous of previous buil...