InnovationQ will be updated on Sunday, Oct. 22, from 10am ET - noon. You may experience brief service interruptions during that time.
Browse Prior Art Database

Reducing Data Loss and Accelerate Re-build in Distributed Storage System

IP.com Disclosure Number: IPCOM000247564D
Publication Date: 2016-Sep-18
Document File: 7 page(s) / 165K

Publishing Venue

The IP.com Prior Art Database


Distributed storage systems typically comprises cache memories that are coupled to a number of disks wherein the data is permanently stored. The disks may be in the same general location, or be in completely different locations. The advantages of this kind of storage system can include improvement of performance, high availability of flexible data path, and protect the systems from disk/module/rack failures. While, most of this kind distributed storage systems is with two copies of data, as one primary data and one backup data, which provides the ability to survive from single disk/ component failure. When one copy failed or damaged, the other copy will be used for a re-building process to keep redundancy of the data block. But with more disks, modules and racks involved in the system, there is a probability that double or more disks and modules failed before the re-build completed. With the usage of storage components years after years, there will be greater probability to encounter this failure. Mostly, this failure will cause data loss or inaccessible. This invention is propose to provide one way to reducing data loss from double or more disks/components failure and provide some fast ways to re-build the storage system.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 52% of the total text.

Page 01 of 7

Reducing Data Loss and Accelerate Re-build in Distributed Storage System

This invention is intended to provide one flexible way to reduce the risk of data loss from multiple disk/component failures, while using as less system storage space as possible. The main idea of this method is to dynamically adjust the policy of the distribution rule of data stripes. And the rule is tiered with vulnerability factor of each data block. The advantage is that it will use very limited system storage space and system spare space for the backup. As the data strip distribution rule changes within this method, the re-build time is also be improved, which is also help to keep the high availability of the system.

Fig4.1.a-Fig4.1.d show the structure and working flow of this method.

a. Fig4.1.a shows an example of the initial status of data block distribution. In this example, there are totally 7 data blocks,distributed in 6 HDD (Hard Disk Drive) disks. Each data block has one main stripe and one backup stripe. The main stripe and backup stripe of the same data block are not in the same disk to ensure data redundancy when the disk is damaged. At first, the vulnerability factor of each disk is the same, i.e. 0. After the system is initialized, the system start to record the vulnerability factor of each disk.


Page 02 of 7

Fig 4.1.a Initial Data Block Distribution

The vulnerability factory for any disk is the product of two factors such as the immediate vulnerability of a specified disk factor multiplied by the associated secondary (e.g., backup) copy, as shown by the following equation:
Fdisk is the total vulnerability of any disk


Page 03 of 7

Where Fmain is the main stripe of data block and Fbackup is the backup stripe of data block. They can be calculated by below equation:

Where the Fdata_category describes the sensitivity associated with each type of operation being performed, Fcommand is the read and write operation, Fresponse is associated with the response fail result, Ftime is the time of failure factor, and Fsuccess is the operation success factor.

Overall, the value of Fdisk is changed from 0 to 1. It represents the probability of the failure of a disk. It can be set as several thresholds to determine the distribution rule change. In this method, T is the current value of the factor of a disk, and Tyellow is set as 0.3, Torange is set as 0.6, and Tred is set as 0.9.
b. System detects the vulnerability factor of each disk. When the value of this factor is reached a threshold, system reschedulethe distribution rule. Different threshold regions are defined with different rules. The rules are defined as below:
i. Yellow Rule: If T< Tyellow, this disk will be marked as 'healthy', and perform the normal behavior. If Tyellow <T< Torange , it wi...