Browse Prior Art Database

An efficient way to evolve accuracy test case of a cognitive computing software system

IP.com Disclosure Number: IPCOM000249106D
Publication Date: 2017-Feb-07
Document File: 9 page(s) / 169K

Publishing Venue

The IP.com Prior Art Database

Abstract

This disclosure is about how to evolve test case in a smart way to improve the efficiency in detecting accuracy code defect of a cognitive computing software system during system testing.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 36% of the total text.

1

An efficient way to evolve accuracy test case of a cognitive computing software system

Background Cognitive computing is the simulation of human thought processes in a computerized model . Software that provides cognitive computing capability is different from traditional software significantly, just list a few:

• The most variant part is the external data source in a cognitive computing software system versus the software’s algorithm in traditional software.

• Data source is changing constantly. • The system has to keep up with the pace of quick changes.

Overview These dynamics causes challenges in system testing, specially the accuracy testing of the cognitive computing system. As a system to think like human and provide advisory, the accuracy is key to its success.

In the traditional software system testing, the test cases are relatively stable where the software is expected to return a consistent result against changing algorithm or new code for introducing new features. In a cognitive computing software system, the software is expected to capture new data from external sources, such as medical journals, drug database etc. As a result, we will see a lot of “failure” in the test result because the expected result is out dated by new development in a particular domain . How should we analyze the “failures” and improve the system’s quality when testing the accuracy of a cognitive computing software system?

This disclosure is about how to evolve test case in a smart way to improve the efficiency in detecting accuracy code defect of a cognitive computing software system during system testing.

Description

Define the major states in cognitive computing system from the system testing perspective

1) Training phase

2

When the system is first developed, the gap between actual result and expected result is big.

2) Stable phase The team analyses the gaps, identifies the defects in the algorithm and test case. Those actions will make the actual result match the expected result. The cognitive system evolves to the stable phase.

3) Evolving phase As the external data change over time and the algorithm is also being improved. The actual result will show small discrepenses from the expected result. This is an essential phase to verify the cognitive system “learns” the changing inputs and generate different output .

Life cycle of a healthy cognitive computing software system from system testing perspective

Cognitive computing system is dealing with moving data, 100% test case pass rate is not the ultimate goal any more. However, being a software system, code error is still inevitable, we need to identify the genuine error.

How to find out code error efficiently in the evolving phase? 1. Create a test case update prediction engine

Test case prediction engine is used to detect external data change and update test case data based on predefined rules automatically.

Test Case Prediction Engine Workflow is as follows: 1) Set up the rules to predict the changes...