Browse Prior Art Database

Original Publication Date: 2001-Sep-01
Included in the Prior Art Database: 2003-Jun-21

Publishing Venue



Disclosed is a method for efficient loading of data from an external file into a database using asynchronous buffered block writes. The method minimizes I/O wait time, and eliminates row-by-row logging. The loader would run as a process separate from the database manager. The loader would read the external file, do the necessary conversions, and write data asynchronously directly to the file or device corresponding to the table to be loaded, thereby by passing the database manager. The asynchronous I/O (CPU-I/O parallelism) reduces the I/O wait time to the absolute minimum, thus enabling the CPU to focus on the time consuming task of doing validity checking, data conversion, and record formatting. Since the creation of the pages containing data is done outside the data manager there is no automatic logging of the data loaded. It is possible, that somewhere in the midst of the data load, a hardware or software failure occurs leading to incomplete execution of the load. If such an incident occurs, one option is to delete the data loaded so far and restart the load job. If the load is a very time consuming one, then it is desirable to take advantage of the data already loaded. However, two problems need to be solved before this can be done: 1. The exact number of complete rows written to disk has be determined.