Browse Prior Art Database

Fast Asynchronous Combinatioral Testing (FACT) method for finding concurrency defects in a software product.

IP.com Disclosure Number: IPCOM000201863D
Publication Date: 2010-Nov-29
Document File: 6 page(s) / 47K

Publishing Venue

The IP.com Prior Art Database

Abstract

Fast Asynchronous Combinatioral Testing (FACT) method for finding concurrency defects in a software product. In many test methods, functionality to be tested is run repeatedly in isolation. Such functionality may involve concurrent execution of multiple-threads and/or processes. If there are concurrency and/or serialization problems within this functionality, such testing approach makes it difficult to uncover them. Without changes in the execution profile, CPU scheduling and/or resource consumption, repeating the test simply does not yield any new timing conditions. As a result serious software defects go undetected. If multiple functionalities that consist of concurrent tasks that are run together the behavior is unpredictable due to timing of execution will prevent referencing the outcome to a known result. This is real time situation and the result is non-deterministic. Known test methods targets single specified functionality. While they are important methods to verify if the functionality really performing what it supposed to do, they lack in verifying concurrency issues that may arise when many individually tested and working functionality runs in parallel The idea is then to spawn multiple FSMs asynchronously and in a random fashion against the same software system. By running multiple FSMs in parallel, racing conditions for shared resources are created which guarantee variation in the execution profile of each FSM. What this achieves is to stress concurrently and serialization mechanisms to the extent where potential malfunctions and deficiencies are more likely to surface.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 26% of the total text.

Page 01 of 6

Fast Asynchronous Combinatioral Testing (FACT) method for finding concurrency defects in a software product.

Disclosed is a process providing a capability to spawn multiple finite state machines (FSMs) asynchronously, in a random order against a same software system. By running multiple FSMs in parallel, embodiments of the disclosed process create racing conditions for shared resources, which guarantee variation in an execution profile of each FSM. Embodiments of the disclosed

p

 rocess stress concurrency and serialization mechanisms to an extent where potential malfunctions and deficiencies are more likely to surface.

Concurrency allows parallel tasks to be performed simultaneously by providing methods for synchronization between processes and data accessed in a computer system. Testing of software systems using concurrency and parallelism typically has issues. For example, in many test methods, functionality tested is executed repeatedly in isolation. Such functionality may involve concurrent execution of multiple-threads and/or processes. When there are concurrency and/or serialization problems within the tested functionality, the testing approach leads to difficulty in uncovering the problems. Without changes in an execution profile,

rocessor scheduling an

                                                      /or resource consumption, repeating a test simply does not yield any new timing conditions. As a result serious software defects may not be undetected.

Common test methodologies used in the software industry today include Unit Testing (UT), Functional Verification Testing (FVT), Regression Testing (RT) and System Verification Testing (SVT). Unit Testing (UT) tests the smallest units of a source code comprising a certain

d

 iece of functionality including error handling. Functional Verification Testing (FVT) validates implemented function against expected outcome and behavior. FVT should include both successful and error outcomes. Regression Testing (RT) is often automated, and validates already implemented functionality has not been regressed or, in any way, affected by subsequent software changes. System Verification Testing (SVT) typically validates all system components including hardware and software are capable of performing according to functional (and, often

p

p

 erformance) specifications under both normal and abnormal (read stress, low resource context, etc.) conditions

When multiple functionalities consisting of concurrent tasks are run together behavior may be unpredictable due to timing of execution, which may prevent referencing an outcome to a known result. The execution is real time situation in which the result is non-deterministic. Known test methods typically target a single specified functionality. While the known methods are important to verify when functionality really performs as required the known methods lack a capability for verifying concurrency issues that may arise when many individually tested and working functionality executes in parallel

The disc...