Browse Prior Art Database

Optimize data migration in a system which mixes local storage devices and Cloud storage

IP.com Disclosure Number: IPCOM000238089D
Publication Date: 2014-Aug-01
Document File: 3 page(s) / 64K

Publishing Venue

The IP.com Prior Art Database

Abstract

This article describe a method which can improve the performance when Cloud is brought into tiering storage system as a new tier

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 52% of the total text.

Page 01 of 3

Optimize data migration in a system which mixes local storage devices and Cloud storage


1. Background

In a tiering storage system, data keeps being moved to or off the better performance storage devices based on its workload. Traditionally, extents are used as the data migration unit. But when Cloud storage is brought into the tiering storage system, we have to meet the performance issue of moving extents between local storage devices and Cloud storage. The extent size in traditional storage system is tens or hundreds of MB, or even GB. Because bandwidth through the Ethernet network is poor, these extent sizes are too big as the unit to move data between Cloud storage and local storage devices.


2. Summary of Main Idea

Cloud storage use objects as the storage units. Comparing with extent, object is much smaller. For example, the object size is only 256KB in the OpenStack storage service: Swift. This gives us the chance to move data between local storage and Cloud storage in objects.

Cloud storage is brought into the tiering storage system as cloud tier. The storage capacity of cloud tier is provided by Cloud Storage, but is still organized by extents. Within the cloud tier, extents are split into objects, and object is used as the unit to move data between local storage devices and Cloud storage.

The main ideas are:
1. When demote data to Cloud storage, firstly pick up the lowest bandwidth extents in the local storage as the candidates , divide them into objects, the least frequently accessed objects are picked to move to Cloud storage.


2. A part of local storage space is used as the Cloud cache. For any object which is to be moved to Cloud storage, it is put in this cloud cache before it is really flushed to Cloud storage. For any cloud object which has IO, it is firstly cached in the Cloud cache before the IO r...