Dismiss
InnovationQ will be updated on Sunday, Oct. 22, from 10am ET - noon. You may experience brief service interruptions during that time.
Browse Prior Art Database

Method to Compute Test Coverage in Complex Computer System Simulation

IP.com Disclosure Number: IPCOM000118496D
Original Publication Date: 1997-Mar-01
Included in the Prior Art Database: 2005-Apr-01
Document File: 4 page(s) / 126K

Publishing Venue

IBM

Related People

Baumgartner, J: AUTHOR [+2]

Abstract

A most prevalent problem in complex system verification is the measurement of test coverage, i.e., measuring how much of the system has been functionally verified in simulation. This problem arises from the system complexity, which prohibits the enumeration of all system states and events that must be verified for correctness. A common way to determine if a system has been satisfactorily verified is to run simulation for a predetermined number of cycles and to stop simulation when the target number is reached and no more bugs are found. This disclosure describes a method by which coverage can be measured in a methodical and intuitive way during simulation of complex computer systems.

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 46% of the total text.

Method to Compute Test Coverage in Complex Computer System Simulation

      A most prevalent problem in complex system verification is the
measurement of test coverage, i.e., measuring how much of the system
has been functionally verified in simulation.  This problem arises
from the system complexity, which prohibits the enumeration of all
system states and events that must be verified for correctness.  A
common way to determine if a system has been satisfactorily verified
is to run simulation for a predetermined number of cycles and to stop
simulation when the target number is reached and no more bugs are
found.  This disclosure describes a method by which coverage can be
measured in a methodical and intuitive way during simulation of
complex computer systems.

      Any system is made of component units and hence system
verification can be thought as verifying each individual unit in the
presence of the rest of the system around it.  If we can determine
that each component unit has been fully verified within a system
configuration, then we can reasonably claim that the entire system
has been verified.  This approach of "divide and conquer" is at the
heart of the coverage measurement that is proposed in this
disclosure.

      Typically, experience has shown that design bugs related to
memory coherency are exposed in system simulation when one or more
transactions happen on the interfaces to a cache controller, where
the transactions are related to the same cache block.  Such
concurrent occurrence of transactions at the interface is generally
called a "race"  or "collision" or "event".  (In this disclosure,
these terms are used interchangeably.)

      Since the control logic in a cache controller must account for
all possible race conditions to happen at all possible system cache
states, it is easy to see that bugs can be introduced by designers
not accounting for certain races.  The coverage methodology described
here is precisely intended to verify that this area is covered.

      The Figure shows a typical shared-bus multiprocessor system
with processors P1, P2,..., Pn and memory cards M1, M2, ..., Mp.  L1
and L2 denote the first- and second-level caches.  For each unit such
as L1 cache, L2 cache, etc., we enumerate all possible race
conditions that can happen at the interfaces.  For instance, if L2
has four ports P1, P2, P3 and P4 connecting it to L1 and another port
P5 connecting to the system bus, then the set of all possible races
for the L2 controller is given by the Cartesian product:
     {t1} x {t2} x {t3} x {t4} x {t5}
  where {ti} is the set of all possible transactions that can happen
on port Pi independent of those occurring in other ports.  In this
case, a race is partly described by the quintuple
     <t1,t2,t3,t4,t5>.

      The transactions are restricted to address the same cache
block.  A transaction that does not address any block is allowed to
be part of any race.  If...