Dismiss
There will be a system update on Friday, May 5th, 6 PM ET. You may experience a brief service interruption.
Browse Prior Art Database

File System Filters: Easily add functionality to file systems

IP.com Disclosure Number: IPCOM000009147D
Publication Date: 2002-Aug-09
Document File: 3 page(s) / 9K

Publishing Venue

The IP.com Prior Art Database

Related People

Wayne Gramlich: INVENTOR [+2]

Related Documents

http://www.faqs.org/rfcs/rfc1094.html: URL [+4]

Abstract

[ IPCOM000000004S originally published 2001-05-07 14:53 UTC ] Disclosed is a simple method for implementing file systems of various types, and optionally combining those implementations for increased functionality. Using the NFSv3 protocol, a user can write a file system that communicates through the TCP/IP networking protocol, which provides a simple bidirectional stream. This makes it easy to develop and debug new file systems; to add new functionality to existing file systems; and to interpose several of these functions in series between the user and the underlying data, in effect making modifications to existing file systems that then become accessible to the operating system. The accompanying text file contains the fully expanded version of this disclosure. Originally, a file system was a direct link between the operating system and the stored file data. This meant that a file system had to contain code to deal with disk sectors, directory structure, caching, and so on. The Networked File System (NFS) exported file data over a network, but programs using the data had to deal with its networking protocol, which introduced new difficulties because the protocol used, UDP, was inherently unreliable and unordered. NFSv3, sometimes called WebNFS, uses a TCP/IP stream instead of UDP. TCP/IP is a much more convenient protocol, and it's easy to parse, monitor, and modify the data, or generate new data to simulate additional files, or even create an entire virtual file system. Although TCP/IP is frequently used for networking, it may also be used by programs on the same computer to talk to each other. Unix operating systems can be instructed to mount (give access to) any file system that conforms to the NFSv3 protocol. To create a new file system, one need only 1) Make a computer export its file system over NFSv3. 2) Run a program (file system filter, or fsfilter) that connects to the NFSv3 port and makes its data available via a second port. 3) Tell the operating system to treat the second port as an NFSv3 file system (for example, option "tcp port=#"). The fsfilter can modify the data passing through it to add any desired functionality. Since the input and output of the fsfilter use the same format, several fsfilters may be connected together, so that each fsfilter has access to the file system data. Although the simplest fsfilter has only one input and one output, an fsfilter may combine data from several different sources, including NFS servers on multiple machines, direct disk access, and other protocols such as HTTP. Some useful applications of fsfilters include: 1) Compress all data being stored, and uncompress it as it is retrieved. 2) Encrypt the data before it is stored, and decrypt automatically as it is read. 3) Provide a trace or dump of the NFSv3 traffic e.g. for debugging purposes. 4) Store each file to two different file systems, as a simple way to provide data backup. 5) In any application where files are stored on several systems with different response times, return read data from whatever system responds the fastest. This may be especially useful for Web "mirror sites" or load balancing. 6) An fsfilter may be used for caching to improve efficiency. 7) Maintaining a version file system, where every open-for-write created a new file with the same name but a version number appended. An open-for-read to a file name without version number would open the latest version of the file. 8) Overlay file system, in which a base file system is read-only, but users want to modify the presented data. As long as a file is not modified, the file system will return the data from the original read-only file. New files (including subdirectories) may be created in any directory. 9) More complicated version control, including concepts of check out, modify, and merge. This is especially useful in a networked multiuser environment. 10) Web file system, in which the fsfilter fetches web (HTTP) data and presents it as files and directories. This is beyond the scope of this disclosure, but is covered in the companion disclosure, "Web File System". This will allow standard unmodified software tools to directly access Web content. 11) File system export, converting file system data to another protocol such as HTTP or WAP. This may be useful, among other things, for passing data through firewalls, and may also be used as a simple web server. 12) Automounting file systems. An fsfilter could mount any file system when it was first accessed, allowing transparent automatic access to any file system. 13) File system translation, for example between NFS and Microsoft's SMB protocol. Keywords: NFS Network File System HTTP WAP Samba SMB NFSv3 WebNFS Tenex TOPS-20 version file system TCP/IP overlay file system filter client server [ 000000004S 04S 4S ]

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 32% of the total text.

Disclosed is a simple method for implementing file systems of various types, and optionally combining those implementations for increased functionality.  Using the NFSv3 protocol, a user can write a file system that communicates through the TCP/IP networking protocol, which provides a simple bidirectional stream.  This makes it easy to develop and debug new file systems; to add new functionality to existing file systems; and to interpose several of these functions in series between the user and the underlying data, in effect making modifications to existing file systems that then become accessible to the operating system.

Originally, a file system was a direct link between the operating system and the stored file data.  This meant that a file system had to contain code to deal with disk sectors, directory structure, caching, and so on.  The Networked File System (NFS) exported file data over a network, but programs using the data had to deal with its networking protocol, which introduced new difficulties because the protocol used, UDP, was inherently unreliable and unordered.

NFSv3, sometimes called WebNFS, uses a TCP/IP stream instead of UDP.  TCP/IP is a much more convenient protocol.  The data carried by the stream is well-ordered and reliable as long as the connection exists; this makes it easy to parse, monitor, and modify the data, or generate new data to simulate additional files, or even create an entire virtual file system.  Although TCP/IP is frequently used for networking, it may also be used by programs on the same computer to talk to each other.  For example, the X Window system uses a TCP/IP stream, even when the program and the window are on the same machine. 

Unix operating systems can be instructed to mount (give access to) any file system that conforms to the NFSv3 protocol.  To create a new file system, one need only 1) Make a computer export its file system over NFSv3.  2) Run a program (file system filter, or fsfilter) that connects to the NFSv3 port and makes its data available via a second port.  3) Tell the operating system to treat the second port as an NFSv3 file system (for example, option "tcp port=#").  The fsfilter can modify the data passing through it to add any desired functionality.  Since the input and output of the fsfilter use the same format, several fsfilters may be connected together, so that each fsfilter has access to the file system data.  The fsfilters may be run as user programs, making them very easy to develop and debug.  Passing data through an fsfilter requires a context switch, but modern operating systems are reasonably efficient at context switches, so this loss of efficiency is tolerable.

Although the simplest fsfilter has only one input and one output, an fsfilter may combine data from several different sources, including NFS servers on multiple machines, direct disk access, and other protocols such as HTTP. 

Some useful applications of fsfilters include:

1) Compress all data being stored,...