Re: NTFS-like streams?

From: Christer Weinigel (wingel@hog.ctrl-c.liu.se)
Date: Mon Aug 14 2000 - 12:30:59 EST


vonbrand@inf.utfsm.cl wrote:
> I read it as "If it can be supported in a sane way, we should do it". This
> is insane, as are all the other schemes I've seen discussed here. And
> "supported somehow" leaves a lot of non-kernel options open: Special tools,
> a compatibility library, ...

So what is the sane way? There are three conflicting requirements for
NTFS-like streams handling:

1. Don't break any existing tools, everything must behave exactly as
   it does today. tar must be able to make a complete backup.

2. Should allow old tools to work on the alternate streams (i.e.
   xv file/thumbnail.xpm).

3. Should not pollute the filesystem with extra files such as
   .AppleDouble, .fork, foo:resources or foo#tar/altstream.

The HFS way fulfills requiremeent 1 and 2 and also means that I can do
"tar cvf ~/foo.tar foo .AppleDouble/foo" and then unpack the tar file
on an ext2 filesystem and have it work sanely. The .AppleDouble way
can easily be extended to work with arbitrarily named files. And of
course it fails requirement 3, and it can be argued that it is ugly to
keep the alternate streams so "far" away from the main stream. It can
also get rather confusing when a user uses non-AppleDouble aware tools
such as mv or cp on an ext2 filesystem, since it will only work on the
default stream, so that the alternate streams can get orphaned or
lost. On a true NTFS file systems, this problem won't exist since
renaming the main stream will automatically rename the alternate
streams too, but on the other hand, suddenly files get magically
removed under the feet of the user. ".fork/file/altstream",
"foo:altstream" and "foo#tar/altstream" are just different ways of
doing this, they all suffer from the same problems. It's also
possible to hide some of these extra filenames but it just means that
one has to come up with a new way of enumerating the alternate
streams.

The "files as directories" way is a big deviation from how things work
today, suddenly one can do opendir/readdir on a file which will
confuse the hell out of some tools and tar won't be able to make a
full backup anymore or be exported by NFS (i.e. wont fulfill no 1).
It will fulfill no 3 and probably no 2 (at least work with those tools
that aren't too smart look at each part of the path to check that
those really are directories).

Another way is to implement a completely new API to access streams,
for example:
    open_stream(fd, name) which returns a file descriptor
    create_stream(fd, name) which returns a file descriptor
    remove_stream(fd, name)
    enumerate_streams(...) which is similar to opendir/readdir
This would work with no 1 and 3 - would not confuse any existing tools
with a file/directory duality, but would also break no 2 by not
allowing existing tools to work with the alternate stream data. And
if one looks at the above functions, they are just a duplication of
the normal open, create and opendir/readdir API's.

I kind of like the "files as directories" idea, but it means breaking
compatibility with all existing Unices. The HFS way is a bit ugly,
but is proven to work and is compatible with just about everything,
but it has its problems with orphaning. The "different-API" model is
clean, but means that just about every tool existing has to be
rewritten to support alternate streams (for extended attributes, I
belive this is the way to go, but for data streams it's not).

Any comments, I've just tried to do a brain dump to see if anybody
agrees with me or if we're still talking past each other.

Are there any other ways of doing this?

  /Christer

-- 
"Just how much can I get away with and still go to heaven?"

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Tue Aug 15 2000 - 21:00:33 EST