Browse Prior Art Database

Compressed Sequential Data On a Fixed Block Device

IP.com Disclosure Number: IPCOM000119609D
Original Publication Date: 1991-Feb-01
Included in the Prior Art Database: 2005-Apr-02
Document File: 7 page(s) / 252K

Publishing Venue

IBM

Related People

Batalden, GD: AUTHOR [+4]

Abstract

Described is a method which allows hardware data compression, i.e., data compression done in an I/O processor, to be used with a fixed block device, such as a disk or diskette. The compression is ideally done by an intelligent I/O processor, but it can also be done by a host computer. Finally, random access to the compressed data records is possible.

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 28% of the total text.

Compressed Sequential Data On a Fixed Block Device

      Described is a method which allows hardware data
compression, i.e., data compression done in an I/O processor, to be
used with a fixed block device, such as a disk or diskette.  The
compression is ideally done by an intelligent I/O processor, but it
can also be done by a host computer.  Finally, random access to the
compressed data records is possible.

      The technique first makes some additions to a high-level file
management system to allow the use of compressed data.  This addition
makes it possible to keep track of which blocks on the media are
associated with the logical records sent to the device. The file
management system already records information about free space
available on the media, the names of data files, and, of course, the
locations occupied by the various files. This information is recorded
in a directory that is located on the media along with the users data
files.  See Fig. 1 for a simple diagram of the media space.

      To allow compression, another attribute is added to each entry
of the directory.  This attribute specifies if the data in the file
is compressed.  Also, another pointer is added in the directory
information.  This pointer points to a list of compression
information.  See Fig. 2 for a simple diagram of the modified
structure.

      The data placed in the compression data area is nothing more
than a list of the compressed record lengths.  Each time a record is
written to the file and compressed, the compressed record length is
saved.  Note that this modification will allow compression by the I/O
Manager (IOM), or by an "intelligent" I/O Processor (IOP) with its
own processing capability, or even by the device itself.  If the I/O
Manager compresses the data before writing to the device, the
compressed length is saved and only the compressed data is sent to
the device.  If the original data is sent to the IOP and compressed
there or in the device, it is only necessary for the IOP to return
the size of the data as actually written.

      The implementation actually used involved compression by the
IOP before sending data to the device.  See Fig. 3 for a diagram of
the data flow for a write from the host to the device.  Fig. 4 shows
a picture of the various data lengths when this write operation is
done. The data transfer starts with a request from the host to write
a number of blocks of data.  The data flows over the bus to the IOP.
Within the IOP the data is compressed and stored in an internal
buffer.  With a variable length device, such as tape, the data could
then flow directly to the device.  However, the complications of a
fixed block device require intervention at this point by the
processor in the IOP.  The data transferred across the SCSI bus
cannot be simply the compressed data.  Instead the hardware requires
that the length be adjusted up to the next block multiple.  This
adjustment is done by...