Re: Implementing Meta File information in Linux (and a note at the end on current reiserfs status)

Hans Reiser (reiser@idiom.com)
Wed, 02 Sep 1998 12:25:09 -0700


Kenneth Albanowski wrote:

> On Tue, 1 Sep 1998, Hans Reiser wrote:
>
> > Theodore Y. Ts'o wrote:
> >
> > > Instead, we're much better off designing a high-level API (implemented
> > > using a replaceable shared library) for storing and retrieving metadata
> > > information (and a common metadata format which both KDE and GNOME
> > > share!!!), and then having a shared library which implements the storage
> > > of said metadata information via some non-kernel, non-FS means. If
> > > later somoene wants to extend the filesystem to provide storing resource
> > > forks, fine we can replace the shared library with one that makes the
> > > non-standard, non-POSIX API calls, but I really don't see any value in
> > > storing such information in this fashion over any one of the myriad of
> > > schemes which don't require filesystem support.
> >
> > This is the structured storage approach. It is hideous. It creates
> > non-symmetric semantics, and it inevitably results in lower performance
> > implementations because it layers one storage allocation system on top of another.
> > Just look at what happened when MS did this.
> >
> > What do you get from your API that you can't get a higher performance
> > solution for in ReiserFS? Even just the solution of a directory called
> > filename.forks is far superior to any structured storage style solution,
> > and filename.forks would not require any changes to namei().
>
> Hans, we can do both! There's no reason the API couldn't be adapted to use
> reiserfs as the storage mechanism (probably requiring a few non-POSIX
> kenel APIs) if available, and other mechanisms if not. The mere existance
> of an API doesn't limit speed, unless it is dramatically dissimilar to the
> underlying semantic -- and I _don't_ think anyone is planning on an MS
> style registry at the moment.
>
> Moreover, we cannot aford to implement something like this without a
> portable library/API. Reiserfs may be a perfect implementation, but that
> won't give us cross-platform support, or networkability. Without those,
> it's going to be a non-starter for many people.
>
> > You would bloat every user application with the need to understand and employ
> > a new glibc API so that you can avoid complicating your FS, and force the
> > application developers to write that API so that you won't have to, and the result
> > will be unnecessarily non-symmetric code that is lower performance.
> >
> > Perhaps I am being unfair in assuming that it would resemble what MS did but.....
>
> First of all, all of this stuff involves hitting the disk, so _any_ C API
> is going to get lost in the wash. (I know that's not true with a fancy
> cache, but it's close enough to true.) Secondly, we seem to be talking
> about file forks at the moment, not MS style registry information.
> They're not necessarily the same thing -- don't forget that the Mac
> doesn't store each individual piece of resource data in a separate fork,
> they all go in one fork, using yet another data format.

I see it as important that the API not perform the aggregation of each piece of resource
data into a single file for the case of ReiserFS. Do we have consensus
on this? Stephen?

>
>
> > Let's just use directories, and make them efficient enough that it works.
>
> Exactly, so we could simply start out by proposing an API that allows
> you to portably treat traditional UNIX filesystems as containing metadata
> (through dot directories, or whatever) that would also apply equally well,
> and more efficiently, to reiserfs.
>
> Moreover, we _must_ have an API to turn any "hidden" data into a flat
> format, or we've just recapitulated the Mac.
>
> > I'll bet you dollars to donuts it went exactly the same way at MS,
> > and the needs were met by writing storage system code that didn't require
> > the help of those FS guys who didn't care, that layered ontop of the FS,
> > that bloated the code, that slowed performance, that reduced code symmetry,.....
>
> And that also needed to work on FAT, SMB, NFS, Lantastic, NTFS, and other
> file systems.
>
> Yes, a new FS is an interesting and useful approach, but it isn't
> instantly available to everyone.

So how about having the API do nothing in the case of Linux, and do lots for other systems?

What do you do when the user wants to access the resource data using utilities like
cp and mv? If you use an API, then what, you emulate namei() with different
implementations for different shells? namei() exists for a reason. We could have
implemented namei() in glibc too (I think there may exist IBM OSes that do the equivalent
of this), but.....

An API that only does something for the case of non-Linux sounds good to me.
I could even live with an API that only does something for the case of non-ReiserFS.

I don't know anything about RMS, but ORACLE uses raw disk for a reason..... To
avoid letting layering give you the worst performance aspects of two storage models
combined.

Hans

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.altern.org/andrebalsa/doc/lkml-faq.html