Browse Prior Art Database

Method and System for Recovering from Partial Writes on a Storage System Using Log Structured Array

IP.com Disclosure Number: IPCOM000248720D
Publication Date: 2016-Dec-29
Document File: 5 page(s) / 70K

Publishing Venue

The IP.com Prior Art Database

Abstract

A method and system is disclosed for creating a transaction-like protocol over a software LSA for recovering from partial writes in a storage system. The new data is written as part of the LSA writes but the references to the old data is not marked as 'Invalid' until after all the new data has been written and acknowledged by the underlying media.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 27% of the total text.

1

Method and System for Recovering from Partial Writes on a Storage System Using Log Structured Array

In complex layered storage systems, user input/output (I/O) operations, specifically writes, can be broken down to smaller I/Os when being issued to lower layers. This causes a potential issue with partial writes as some of the smaller I/Os succeed while others fail due to a hardware problem with the underlying media, leading to data corruption. In such a case, the I/O failure is notified as an appropriate error to the user, which cannot assume anything about the data.

However, when the write operation is expected to be atomic or when it is a part of a write-in-place on a log structured array (LSA), the corrupted data caused by the partial write may cause other data to be corrupted as well, unless special measures are taken to update the metadata for the whole LSA page within which the corrupted data resides.

Disclosed is a method and system for creating a transaction-like protocol over a software LSA for recovering from partial writes in a storage system. The new data is written as part of the LSA writes but the references to the old data is not marked as 'Invalid' until after all the new data has been written and acknowledged by the underlying media.

The software LSA of the method and system is implemented as a unified stack that enables the use of low-cost Solid State Drives (SSDs) and Shingled-Magnetic Recording (SMR) Hard Disk Drives (HDDs) is existing systems.

The figure below illustrates an example of software LSA in the kernel for managing SSD and SMR disks.

Figure

2

Software LSA, for low-cost SSDs, intercepts the user I/O requests and remaps the I/O requests in ways that transform the user access pattern to be as efficient as possible for the underlying storage medium.

The various technologies used by software LSA for low-cost SSDs can be, but need not be limited to, log-structured data placement, redundancy without performance penalties, heat segregation, garbage collection, storage virtualization, in-memory caching, throttling, data reduction and Remote Direct Memory Access (RDMA) interface.

The log-structured data placement is to implicitly force the device controller to not do any garbage collection internally. This is achieved by ensuring that software LSA will always write data to the SSDs using large sequential I/O operations. In particular, as write I/O comes down to software LSA from the user, software LSA packs the written data into large segments. Each segment is typically a multiple of the logical erase block size of the underlying device. Software LSA always writes full segments to storage and, effectively, overwrites multiple logical erase blocks in the underlying device. As a result, when the underlying device attempts to perform garbage collection, all the data in a garbage-collected logical erase block are found to be invalid and no data relocation is required.

Further, software LSA performs garbage collection at the...