Browse Prior Art Database

# Tight and Automatic Range Checking for Fixed-Point Computations

IP.com Disclosure Number: IPCOM000021498D
Publication Date: 2004-Jan-21
Document File: 3 page(s) / 81K

## Publishing Venue

The IP.com Prior Art Database

## Abstract

Disclosed is a method for a new instrument that measures the ranges of intermediate variables of linear transforms effected in fixed-point arithmetic. Benefits include a solution that works with all affine transforms.

This text was extracted from a Microsoft Word document.
At least one non-text object (such as an image or picture) has been suppressed.
This is the abbreviated version, containing approximately 52% of the total text.

# Tight and Automatic Range Checking for Fixed-Point Computations

Disclosed is a method for a new instrument that measures the ranges of intermediate variables of linear transforms effected in fixed-point arithmetic. Benefits include a solution that works with all affine transforms.

## Background

Assessing the numerical range of intermediate variables or states is an important task in fixed-point computation, which ensures proper operation. Fixed-point computations are used widely in many areas, including: telephony, electronic gamming, digital photography, and video. For example, in telephony, it is typical that a windowing operation has to be applied, which involves multiplications by coefficients less than 1 in magnitude; thus it reduces the numerical range of intermediate variables that correspond to higher frequencies. Knowing what this reduction is can potentially lead to a lower data width requirement. On the other hand, an over-aggressive reduction of data may lead to internal overflow. The result of overflow is wrong numerical values that manifest themselves as an erratic voice quality.

In the area of application specific integrated circuit (ASIC), hardware designers typically carry out manual analyses on the numerical ranges of every intermediate variable to optimize the hardware requirements on the data path and arithmetic units. This process is time consuming, error prone, difficult to validate, and usually depends on hardware that is designed for only a few specific tasks.

In the area of fixed-point computational software, an increasing amount of development targets general purpose hardware. This is because a new generation of high-performance fixed-point processors, such as the Intel XScale platforms, are being used on devices such as the latest generation of cell phones, pocket computers, and other handheld devices. In this case, there are only a few choices for data widths (i.e. 16, 32, and 64 bits). However the need to assess numerical ranges is still crucial. Software developers typically scale up their data ranges as much as possible to increase numerical accuracy (thus increasing quality of computed results) without over owing these fixed data widths. Therefore, the general problem is that of assessing intermediate numerical ranges of a fixed-point computational procedure.

## General Description

The disclosed method is based on three ingredients: a model of tight range bounds on all intermediate states (i.e. variables) during a fixed-point computation of affine transforms, the operator overloading facility offered by modern computer language, and a mathematical characterization of the evolution of the range bounds.

Every step in a computer program produces a computed value, y-intermediate, that approximates an affine transform of the independent input varia...