Browse Prior Art Database

# Numeric Value Determination

IP.com Disclosure Number: IPCOM000089333D
Original Publication Date: 1977-Oct-01
Included in the Prior Art Database: 2005-Mar-04
Document File: 3 page(s) / 81K

IBM

## Related People

Kirkman, RK: AUTHOR

## Abstract

An improved method is provided for finding numeric values such as determining roots. The improvement resides in the use of bit testing, instead of digit testing, to determine if a digit is part of the results.

This text was extracted from a PDF file.
At least one non-text object (such as an image or picture) has been suppressed.
This is the abbreviated version, containing approximately 53% of the total text.

Page 1 of 3

Numeric Value Determination

An improved method is provided for finding numeric values such as determining roots. The improvement resides in the use of bit testing, instead of digit testing, to determine if a digit is part of the results.

The flow chart shown above represents the logic to find a numerical value to meet a given condition as follows: 1. f is a strictly increasing one-to-one function whose domain and range are the set of whole numbers. 2. For a whole number y, find the whole number x such that f(x) </=y and f(x+1) > y.

A field (result) to contain the value x is initially set to zero. Using the binary nature of the representation of numeric values in a field, the value of x is determined by setting and testing bits of the bytes in the result field. Proceeding from highest to lowest order bytes and from highest to lowest order bits within each byte, the value x is determined as follows: 1. Set the bit to 1 in the result field. 2. Test the result field by computing f(x). 3. Set the bit to 0 in the result field if, and only if, f(x) > y. 4. Repeat Steps 1-3 with successively lower order bits until the end of the result field is reached or until f(x) = y. When the process is complete, the value of the result field is the desired x.

This algorithm is useful in performing calculations in cases where there are no hardware instructions to achieve the result easily. For example, if f(x) = x/2/, then for a whole number y, the square root of y can be found.

Let y = 0010 1101 1000 0010 1100 0110 (binary) (2,982,598 decimal). The result field then can be no more than two bytes long. Begin with it set to all zeros and proceed from highest to lowest order, setting and testing bits. The following chart illustrates the process.

If a given bit being 1 causes f(x) t...