Browse Prior Art Database

Adaptive Locality Configuration

IP.com Disclosure Number: IPCOM000249478D
Publication Date: 2017-Feb-28
Document File: 2 page(s) / 21K

Publishing Venue

The IP.com Prior Art Database

Abstract

Disclosed is an adaptive locality configuration algorithm that changes its behavior and sets the number of locality memory devices within a pre-defined range, based on information available at the time it is run.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 52% of the total text.

1

Adaptive Locality Configuration

Writing to nearby locations in the namespace range of all possible slice names can have the advantage of improving sequential access to the underlying memory devices within each disperse storage (ds) unit, and is termed "achieving locality". Therefore, restricting the selection of names to limited ranges can lead to more efficient and more performant access. However, forcing all writes to a single area of the namespace also limits the total number of ds units and memory devices involved. There are therefore significant tradeoffs to be made in selecting the number of local areas used, how frequently the areas are changed, how many are used per ds processing unit, how much synchronization/cooperation there is on agreeing on local ranges between different ds processing units, etc.

As an example, a statically defined locality setting might recommend selecting and writing to three locations at a time for a period of one minute. This static configuration, however, does not take into account the number of stripes, memory devices, and disperse storage units (ds unit), or ds processing units, so it does not evenly spread the load across the entire set of ds units as it should. As a result, all Input/Output (IO) loads that hit the system affect a very small number of memory devices, which could simply push those devices too hard.

A possible resolution of this problem is to change to spread the load over 48 locations rather than three, which has the desired effect of spreading the initial surge or IO requests over a much larger population of memory devices. However, there are also some negative impacts. In data source name (DSN) memories with a small number of ds units or total number of memory devices, writing to 48 locations at once can disrupt the benefits of locality, by causing each memory device to have to access multiple distinct locations at once.

Other relevant variables aside from the size of the system include access patterns, workflows, object sizes, segment sizes, storage formats that are used (for example, File Slice Storage (FSS) vs Packed Slice Storage (PSS), etc.). An ideal algorithm for optimizing locality patterns would dynamically adapt to all of these variables and more.

The idea proposed is an adaptive algorithm that changes its behavior and sets the number of locality memory devices within a pre-defined range, based on information available at the time it is run. The trigger mechanism might be information about IO load, throughput, latency, object size, or any other parameter or computational resources available, the number...