Browse Prior Art Database

System for Reducing the Disk Space and Time Required for Processing Very Large Files on Processing and Tape Archiving Systems

IP.com Disclosure Number: IPCOM000117967D
Original Publication Date: 2005-Mar-31
Included in the Prior Art Database: 2005-Mar-31
Document File: 2 page(s) / 42K

Publishing Venue

IBM

Abstract

Disclosed is a combination of a file archiving system and a new stackable file system. This combination reduces the delays and the real disk space requirements involved in writing, archiving, restoring, and reading very large disk files.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 52% of the total text.

Page 1 of 2

System for Reducing the Disk Space and Time Required for Processing Very Large Files on Processing and Tape Archiving Systems

This combination of a file archiving system and a stackable file system makes it possible to put TeraByte Files in a GigaByte Filesystem, and efficiently use disk space at processing and archiving systems. It also reduces idle time at the processing system.

This combination makes it possible, without application program change:

For a program to process what it perceives to be very large (>1TB) local files, even though the system the program runs on does not have that much disk storage.

To stage the file data in sections to/from archiving systems without tying up sufficient disk space to hold the entire file at either the processing or archive system.

For archiving to start even before a file has been completely written by a program.

For a program to begin to read very large (e.g. terabyte) files, even before they have entirely arrived from the archive system.

The components of the combination are a file archive system, such as Tivoli Storage Manager and a new file system for client machines. The file system is referred to here as the "blivet" file system.

The blivet file system is in the category of a "stackable" file system, meaning:

It appears to applications as a file system.

It uses another file system "stacked underneath" it.

Blivet is a stackable file system that segments files and can interface with archiving systems.

It makes the segmentation and segment handling transparent to the processing application program and the archiving system, requiring no change to the processing programs nor to the archive system.

Blivet segments each big file into many smaller ones. For example, the 4 gigabyte file Experiment becomes Experiment.segment00000000000000, Experiment.segment00001000000000, Experiment.segment0000200...