Browse Prior Art Database

LOCATION ZERO REFERENCE DETECTION MECHANISM

IP.com Disclosure Number: IPCOM000016437D
Original Publication Date: 2001-Sep-01
Included in the Prior Art Database: 2003-Jun-21
Document File: 1 page(s) / 44K

Publishing Venue

IBM

Abstract

Disclosed is a method for efficient loading of data from an external file into a database using asynchronous buffered block writes. The method minimizes I/O wait time, and eliminates row-by-row logging.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 52% of the total text.

Page 1 of 1

LOCATION ZERO REFERENCE DETECTION MECHANISM

    Disclosed is a method for efficient loading of data from an external file into a database using asynchronous buffered block writes. The method minimizes I/O wait time, and eliminates row-by-row logging.

The loader would run as a process separate from the database manager. The loader would read the external file, do the necessary conversions, and write data asynchronously directly to the file or device corresponding to the table to be loaded, thereby by passing the database manager. The asynchronous I/O (CPU-I/O parallelism) reduces the I/O wait time to the absolute minimum, thus enabling the CPU to focus on the time consuming task of doing validity checking, data conversion, and record formatting.

Since the creation of the pages containing data is done outside the data manager there is no automatic logging of the data loaded. It is possible, that somewhere in the midst of the data load, a hardware or software failure occurs leading to incomplete execution of the load. If such an incident occurs, one option is to delete the data loaded so far and restart the load job. If the load is a very time consuming one, then it is desirable to take advantage of the data already loaded. However, two problems need to be solved before this can be done:

1. The exact number of complete rows written to disk has be determined.
2. The line number of the input corresponding to the rows so loaded has to be determined.

The loader takes the following approach. At a predetermined checkpoint time, force all pages (which were written asynchronously but are still in system buffer) to disk. Force the system page (the page which contains the column descripti...