Improved Scenario Verification Test Process
Original Publication Date: 2005-Feb-24
Included in the Prior Art Database: 2005-Feb-24
This is a new business process to Scenario Verification Test (SVT), built upon the IBM customer satisfaction characteristics of CUPRIMDSO, combined with the Persona concept of Interaction Design, all wrapped around realistic customer scenarios. The CUPRIMDSO concept is certainly not new at IBM. However, some test organizations have lost sight of its valuable guidance. This methodology starts with those characteristics and builds the test around them. It also uses the Persona concept, as authored by Alan Cooper in ?The Essentials of Interaction Design?. The scenarios emulate how the end-user persona would actually behave while using the product. Finally, to bring the scenarios to life, the process designs several basic categories of tests, with numerous scenarios underneath. Each one simulates actual customer experiences, as seen by the various personas, and verifies the CUPRIMDSO characteristics that customers would ultimately evaluate.
Improved Scenario Verification Test Process The following paper describes the new approach to scenario verification test.
From CUPRIMDSAPI to Personas A New Approach to Scenario Verification Test
When our team set out to improve our scenario verification test methodology, we looked to the industry
for clues to make our testing more effective. We found a couple concepts that seemed promising
1. IBM's CUPRIMDSO customer satisfaction characteristics
2. Persona concept, as authored by Alan Cooper in "The Inmates are Running the Asylum".
The CUPRIMDSO concept is certainly not new at IBM. IBM has been structuring its customer satisfaction surveys around these characteristics, or metrics, which represent various critical aspects of the product solution. We took the IBM characteristics, deleted "overall" because it was more appropriate for a customer satisfaction survey, and then added 3 characteristics that did seem critical to our testing. Hence, our new version of CUPRIMDSO turned out to be CUPRIMDSAPI:
Characteristic Description Capability Functionality of product as compared to the requirements
Integrity of the available functionality
Usability Ease of learning important tasks of the product
Ease of executing the task
How intuitive is the product Performance Hardware/resource requirements to effectively run product
Reliability Freedom from operational failures
Mean time between failures; Severity / impact of failures Installability Ease of installing product and configuring for use
Required skill level to complete installation Maintainability Ease of correcting problems
Ease of installing fix packs or patches
Documentation Hardcopy / softcopy publications and guides; On-line help screens
Ease of finding relevant information in existing documentation Effectiveness of this information Serviceability Ease of identifying and diagnosing problems
Ease of conveying these problems to product's support center Availability Ability of the product, as a whole, to remain available for use.
Poor reliability affects availability, as does scheduled maintenance and negative effects of long-running jobs Personas Use of product by specific customer classes (administrator, operator, end-usage customer, etc.)
Interoperability Ease of integrating with other products
Ability to correctly receive, process, or transmit data to other products
To bring the scenarios to life, these basic characteristics are then applied to high-level categories of tests. These categories can be applied to virtually any kind of test - be it software or hardware.
* Installation / Migration Scenarios
o Install or Migrate test environment to new level, using official documentation
o Simulate actual customer procedures and time patterns
o Use documented hardware/software environments rather than scaled-down servers
o Use actual customer data whenever possible
* Typical Day Scenarios o Typical customer functions, as run b...