Browse Prior Art Database

Effective Method to Migrate Data from On Prem to Cloud using DashDB Disclosure Number: IPCOM000247801D
Publication Date: 2016-Oct-06
Document File: 2 page(s) / 29K

Publishing Venue

The Prior Art Database


We propose a system and method to compress the data at the source and move the data in to the cloud thereby reducing the overhead of compressing at the cloud to enable faster migration to cloud. The method handles both fresh migration and updating data to the already migrated data.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 51% of the total text.

Page 01 of 2

Effective Method to Migrate Data from On Prem to Cloud using DashDB
Enterprises are moving from on prem to cloud to get the benefits of hassle free infrastructure and to rapidly build capabilities. A cloud service is needed to provision such migration to start moving data from any source to cloud and enrich the applications with enhanced capabilities in the cloud. There days there are cloud based databases which store data in compressed format. One such database is IBM dash db. It stores the data in a compressed format. When data is to be migrated from on-prem to such databases, the typical tools will move the data to the cloud and then compress it. THere are a few tools which first compress the data on-prem, move it to the cloud, un-compress it and then load it into the database which again compresses the data.

Such an approach is very inefficient and cumbersome. We propose an approach which addresses this problem.

The detailed solution is explained below:

Enterprises may fall in to two categories when the data is being moved from on-prem to cloud:

1. New Migration: Entire data is being moved from on-prem to cloud for the first time. In this type of migration, all the data has to be migrated to the cloud.

2. Updated Migration: The migrated data is already on the cloud. There is some data that has changed at the on-prem and now it has to be migrated and merged with the existing data in the cloud.

The method is explained by considering the above two cases:

New Migration:

In this case of migration, the data source on the on-prem can be any database like Cloudant, Oracle. The data format need not be same as the DashDb. If the entire data is moved to the cloud, then compressed and stored in the database will take too much time for the migration process and it is cumbersome for the enterprises . Hence we follow the steps below to efficiently store the compressed data:

1. In present article the target database provides a library which will includes the compression algorithm. This library is made available for download using a REST API.

2. Our approach runs on a server inside the firewall (on-prem). It first connects to the target database using the REST API and downloads the jar file.

3. It then retrieves the data from the source database. It then compresses the data using the compression algorithm present in the jar file.

4. The compressed data is then moved to the cloud.

5. The target database provides a new loading mechanism for directly loading compressed data into the database. Our approach makes use of the new API. In this API the database skips the compression step and loads the data directly into it storage.

The above approach avoids a lot of redundant compression/decompression an...