Dismiss
InnovationQ will be updated on Sunday, Oct. 22, from 10am ET - noon. You may experience brief service interruptions during that time.
Browse Prior Art Database

A Method for Measuring Product Readiness in Relation to Test Event Complexity

IP.com Disclosure Number: IPCOM000012892D
Original Publication Date: 2003-Jun-06
Included in the Prior Art Database: 2003-Jun-06
Document File: 2 page(s) / 59K

Publishing Venue

IBM

Abstract

In software verification, testers are constantly challenged by project management to accelerate their work efforts to get to the marketplace quickly and with high levels of quality. There are many factors that are considered when evaluating whether a product is ready for customer use. Those factors include, but are not limited to, testing execution progress, early customer program use and implementation progress, service and support readiness, and the confirmation of performance expectations. Another very important aspect of product readiness is the validation that the product can operate under extreme levels of stress and complexity. Incorporating complexity into the readiness assessment, the metric described in this paper points out that although the majority of planned tasks may have been executed, the difficult tasks are yet to be attempted This is a change from the typical method of comparing tasks completed against tasks planned and using the resultant percentage as a readiness indicator. This paper outlines a method for measuring progress in the areas of product volume and functional complexity with the goal of providing project management with key progress checkpoints to make product availability decisions. The formulated combination of volume and complexity can provide a value with which to measure system level readiness of the product. Test owners can incorporate values into the readiness factor for constructs that influence the complexity of the test event. These constructs might include repetition of workload execution, environmental considerations, or platform genealogy.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 52% of the total text.

Page 1 of 2

A Method for Measuring Product Readiness in Relation to Test Event Complexity

  Introduction - Problem Statement Although load and stress activities are important to the overall progress of testing on software products, it is always difficult to 'measure' when the product or system under test is ready for customer workloads. Getting the product itself into the hands of customers, so that IBM can benefit from the feedback of the customer, is a critical aspect of software development in today's fast-paced Information Technology industry.

Test event planning usually takes into consideration the number of scenarios to be executed, over a given period of time, to determine readiness. This method does not take into account that the work left to perform may be the most difficult and complex tasks planned within the test event. The problem is that there is no systematic way to measure volume and complexity in terms that provide progress indicators. What is needed is methods of projecting when, during the course of a test event, specific milestones occur based on the actual readiness of the product. These milestones may include entrance to another test event, early customer involvement, and general availability of the product or solution. Additionally, understanding the amount of complexity that has been proven at any given point in the test event, test owners can project how much work is left to perform before an upcoming milestone is reached. Solution - The fundamental aspect of the metric outlined here is the assignment of relative values for volume, complexity, and activity. This metric is primarily used during the test planning phase. Test owners assign values to milestones in order to determine either when that milestone will occur, or what work needs to be performed prior to the date of a given milestone. Volume, complexity, and activity values are set when the scenario is written, as determined by an agreed upon convention within the test organization. As testing progresses and scenarios are executed, the values accrue. The test owner can then plot test event progress against projections. The product result of volume, complexity, and activity is a measurement value that will be termed, from this point on, as Amplitude (A). By weighting the volume, complexity, and activity values as a percentage of a projected goal amplitude, it is possible to allow these values to have different relative meanings, expressed as a percentage of the goal. A test owner will predetermine the goals and weights of the test event. Each milestone is assigned a value which is a percentage of the goal amplitude of the test event. For example: A Goal Amplitude ( Ag
) of a test event may be set at one thousand, the relative weight of the volume of work ( Wv ) may be set at fifty percent, the relative weight of the activity(Wa ) may be thirty percent and the relative weight of the complexity of work ( Wc ) may be set at twenty pe...