Browse Prior Art Database

Fast Numerical Conversion From Packed Decimal to IEEE Double (Floating Point)

IP.com Disclosure Number: IPCOM000121537D
Original Publication Date: 1991-Sep-01
Included in the Prior Art Database: 2005-Apr-03
Document File: 2 page(s) / 93K

Publishing Venue

IBM

Related People

Cross, MR: AUTHOR [+2]

Abstract

In database operations it is often necessary to convert numbers stored or defined as Packed Decimal to IEEE Double format. In the OS/2 Extended Edition Database Manager this conversion has been done using a compiler function requiring two steps. First the Packed Decimal was converted to an ASCII string, and then a call was made to a C/2 compiler library function to convert the ASCII string to a double. In order to gain more code portability (with a view to other platforms), a numerical algorithm is needed that avoids reliance on a library. For better performance it also should avoid the overhead of an intermediate conversion to ASCII. It also should be predictable; that is, it should give the same results on different platforms and operating systems.

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 52% of the total text.

Fast Numerical Conversion From Packed Decimal to IEEE Double (Floating
Point)

      In database operations it is often necessary to convert
numbers stored or defined as Packed Decimal to IEEE Double format.
In the OS/2 Extended Edition Database Manager this conversion has
been done using a compiler function requiring two steps. First the
Packed Decimal was converted to an ASCII string, and then a call was
made to a C/2 compiler library function to convert the ASCII string
to a double. In order to gain more code portability (with a view to
other platforms), a numerical algorithm is needed that avoids
reliance on a library. For better performance it also should avoid
the overhead of an intermediate conversion to ASCII. It also should
be predictable; that is, it should give the same results on different
platforms and operating systems.

      Disclosed here is a purely numerical algorithm that will appear
in future releases of the OS/2 Extended Edition Database Manager.  It
makes no library calls, is fast, and is predictable. This algorithm
assumes the following standard definitions:
   1. A byte consists of eight bits.
   2. A LONG INTEGER (4 bytes in the IBM C/2 Compiler) has the
capacity to contain the value 99999999. This is well below the
maximum capacity of 2147483647 for the LONG INTEGER of the IBM C/2
Compiler.
   3. A DOUBLE is as defined in the IEEE and SAA standards.
   4. Packed DECIMAL is as defined in SAA standards.

      Following is a description of the Packed DECIMAL which this
algorithm converts to FLOAT. The Packed DECIMAL can have a precision
(total number of digits) of up to 31. It may have any number of
fraction digits that does not exceed its precision. Its sign is in
the RIGHT nibble of its RIGHTmost byte.  Each byte prior to the
RIGHTmost byte contains two digits, one per nibble with values of 0
through 9.  The exception is that in the case of a DECIMAL with even
precision, the LEFT nibble of its first byte is not part of the
DECIMAL.

      Following is how the algorithm works.  The float (result) will
be accumulated in fval. The algorithm skips leading zero bytes of the
DECIMAL. As bytes are skipped, precision is reduced accordingly. In
case of even precision, if the first byte is zero, reduce precision
by 1 instead of 2. If the precision of the remaining dig...