Browse Prior Art Database

A method to migrate large data in multiple phases

IP.com Disclosure Number: IPCOM000174098D
Original Publication Date: 2008-Aug-26
Included in the Prior Art Database: 2008-Aug-26
Document File: 2 page(s) / 16K

Publishing Venue

IBM

Abstract

A new method is developed to migrate large data between databases. It divides the whole migration into multiple phases according to the length of each data field and also provides a new technique to preserve the data integrity at each phase.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 53% of the total text.

Page 1 of 2

A method to migrate large data in multiple phases

The data migration becomes a very important part of a company's IT system upgrade because of the globalization driven by Internet techniques. However, the migration task is often time-consuming and error prone, especially when it involves with large data. Our solution presents a novel way to migrate large data within or between databases.

Currently, there are two categories of methods a user can use to migrate data. One is to use import/export tools provided by database vendors. Another is to use a stand-alone tool that copies data between databases.

The first kind of methods introduces many I/O operations when creating and reading dump files whose formats are very difficult for a user to manipulate or customize during migration. For example, a user must know details of dump files, if he/she wants to migrate the data from a native encoding column type to a Unicode one. The second group of methods also needs extra network communication between the client tool and servers,

which is much slower and less reliable than disk I/O.

All these methods treat different kinds of data in the same way and execute the major task in one step,

which will cause database servers to exhibit poor response time when handling large data and,

sometimes, make the migration impossible within a required time window.

In our method, the data is migrated in multiple phases according to its size. Our method also improves the migration performance by creating direct connections between database servers. A user can manipulate data using standard SQL statements through direct connections between the source and target databases.

A direct connection between database servers can be implemented on different techniques provided by database vendors. For example, on DB2,

we can use the combination of Wrappers and Nicknames

to create such a connection that is optimized by database servers and much faster than any connection created from a stand-alone tool.

From our analysis on customer's data,

we find large data types, such a...