Browse Prior Art Database

Method for visualising confidence and risk associated with test results

IP.com Disclosure Number: IPCOM000145217D
Original Publication Date: 2007-Jan-10
Included in the Prior Art Database: 2007-Jan-10
Document File: 2 page(s) / 56K

Publishing Venue

IBM

Abstract

Disclosed is a method to show the age of test result data in a cumulative bar graph (one were the most recent results for all tests are shown for each build, even results which weren't executed against that build) by colouring the section of the bar which corresponds to certain result in a shade or intensity of colour proportional to its age. When the data is sorted appropriately the effect is to have a faded bar-chart with the pattern of fading giving a qualitative indication of the reliability of the results being viewed.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 52% of the total text.

Page 1 of 2

Method for visualising confidence and risk associated with test results

A simplistic view of the testing discipline is that all tests are executed every time a new version of the system under test (SUT) is produced. Traditional reports of test results typically show histograms or line graphs showing the number of attempted, passed and failed test cases run against consecutive SUT versions.

    Often, however, owing to resource or time constraints, not all tests are run against every version of the SUT. So for a given version of the SUT not all tests are logged as attempted. However, it may well be that the test was run on the previous version of the SUT, and we should therefore have a reasonable expectation that the test would still be successful. This confidence in the test result would diminish over time as more changes are made to the SUT.

    Currently there is no known standard for conveying this information in a convenient and succinct manner.

    This paper describes a technique to show the age of test result data in a bar graph by colouring the section of the bar which corresponds to certain result in a shade or intensity of colour proportional to its age. When the data is sorted appropriately the effect is to have a faded bar-chart with the pattern of fading giving a qualitative indication of the reliability of the results being viewed.

    As an example, consider software test results accumulated over time. The graph shown shows test results for a particular product supplied by build. The green faded section shows passed tests. The red fading section shows failed tests. The grey section is tests never run at that point. The white sections are effectively very-faded green. The failed tests on the other hand always retain at least the palest degree of red so that the need to resolve them is never forgotten.

%age of all tests

100%

50%

0%

Build i Build j

Consider build 1 - the first build ever produced. About 25% of the tests have

Build 2 Build 3 .....

Build 1 Product Builds

Page 2 of 2

been run and passed, because the tests were run on this build the bar is bright green (the highest intensity). About 10% of all known tests have failed on this...