Browse Prior Art Database

Software Test Coverage Measurement

IP.com Disclosure Number: IPCOM000118052D
Original Publication Date: 1996-Aug-01
Included in the Prior Art Database: 2005-Mar-31
Document File: 4 page(s) / 127K

Publishing Venue

IBM

Related People

Taylor, JL: AUTHOR

Abstract

In testing software, it is required to measure the effectiveness of a set of test cases at exercising a piece of software under test. Data needs to be collected on which instructions/paths in the code have actually run - if an instruction has not been executed, it can not have been tested.

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 45% of the total text.

Software Test Coverage Measurement

      In testing software, it is required to measure the
effectiveness of a set of test cases at exercising a piece of
software under test.  Data needs to be collected on which
instructions/paths in  the code have actually run - if an instruction
has not been executed, it can not have been tested.

      Present non-intrusive techniques include specialized hardware
that monitors the instruction execution of the processor running the
program under test conditions.  The hardware generates a table of
addresses that have/have not been executed.  This hardware is
expensive and specialized - and is typically difficult to develop and
maintain.

      As processors have grown faster and more sophisticated, it has
become increasingly difficult to monitor and record the internal
execution sequence of the processor.  Processor features, such as
instruction prefetch, conditionally executed instructions, parallel
execution and instruction caching make this approach ever more
impractical.  Furthermore, the expense of the test hardware limits
the number of such devices available to a code development/test team.

      A cheaper technique involves the insertion of trace points
into the test code.  This can be done manually, or in some cases
automatically, at compile time.  The trace points must be carefully
chosen to give sufficiently fine granularity of data, while not
slowing the program so much that it ceases to work as specified.

      At the cost of some Random Access Memory (RAM) space on the
target system, and some one-off changes to the run time library, the
technique described here achieves the finest possible data
granularity (individual instructions) while reducing the execution
overhead for the processor to zero on second and subsequent
executions of any given  program location.

      The improved technique invokes start-up code which forms the
initialization part of a compiler's run time library.  It is an
"invisible" part of the program, in that there is no source required
from the writer of application code.  The start-up code is
responsible for setting the initial state of the program before
calling the application code entry point.  Functions that it performs
include the setting up of the stack, heap and the initialized and
zeroed data areas.  Source for the start-up code is normally provided
with the compiler, and can be customized as necessary for a
particular target system.

      The start-up code, which is not regarded as forming part of the
application code, performs the following:
  1.  Make a copy of the application program in some backing
       store (typically RAM which may be fast or slow).
  2.  Install an interrupt handler for a given software interrupt
       (which we shall call the coverage interrupt).
  3.  Overwrite the entire application CODE (but not data) with
       coverage interrupt instructions.  These instruction...