Browse Prior Art Database

Fast Data Integrity Verification and Data Recovery for Compressed Objects

IP.com Disclosure Number: IPCOM000241748D
Publication Date: 2015-May-28
Document File: 2 page(s) / 62K

Publishing Venue

The IP.com Prior Art Database

Abstract

Disclosed is a method to reduce the data scrubbing time for a compressed file system by both lowering the number of blocks to be scrubbed (i.e. scrub only valid blocks) and use error-detecting code verification instead of decompression. This also reduces the Central Processing Unit time needed to decompress the data for invalid blocks.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 59% of the total text.

Page 01 of 2

Fast Data Integrity Verification and Data Recovery for Compressed Objects

The explosion of data, both in motion (i.e. network) and at rest (i.e. storage), increases the need for better compression solutions. Currently, the networking industry uses compression to lower the network bandwidth requirements for growing portions of the internet traffic. Recently, the primary storage industry started using compression to lower the exponentially growing storage capacity requirements.

Data scrubbing is an error correction technique that uses a background task to periodically inspect main memory or storage for errors, and then correct detected errors using redundant data in form of different checksums or copies of data. Data scrubbing reduces the likelihood that single correctable errors will accumulate, leading to reduced risks of uncorrectable errors.

Data scrubbing for a compressed file-system is a slow process that requires high Central Processing Unit (CPU) consumption and Input/Output (IO) volume. Data integrity has to be verified via decompression. A compressed file

system has metadata which must be synchronized to the data after corruption detection (metadata has to be repaired).

The novel contribution is a method to reduce the data scrubbing time for a compressed file-system by lowering the number of blocks to be scrubbed (scrub only valid blocks). This also reduces the CPU time needed to decompress the data.

To reduce the CPU load, the method uses error-detectin...