Re: (reiserfs) Re: Summary of how linux can best avoid the need for streams

Richard Gooch (rgooch@atnf.csiro.au)
Thu, 1 Jul 1999 16:32:49 +1000


Albert D. Cahalan writes:
> Richard Gooch writes:
> > All I am saying is that for some kinds of data (such as a single,
> > large growing component and any number of small maybe growing
> > components), storing it in a special file is fine. It's been done and
> > it works well. I have already said that this case does not apply in
> > general.
>
> Why is it fine just this once? You wanted to manipulate individual
> components with normal tools, but clearly you can't. You need special
> tools to insert and extract data. If anything, this is _worse_ than
> a multi-forked file. You will need a separate tool for each app.

The origin of my comment is a rebuttal of the statement that someone
(I don't recall who) made that "multiple data streams in a single
(real) file will be inefficient". I've have refuted that assertion. I
have evidence (existing real-life applications and data that)
disproves that assertion.

I agree that "multiple large, growing data streams in a single (real)
file is inefficient". But that is a very different statement from the
one above. I am quite willing to concede that a word processor
document format falls into the latter category.

> > No, I didn't suggest that. Again, I favour optional extensions to new
> > and existing tools, so that you can view albods as either atomic or as
> > conventional directories. Make the default behaviour configurable on a
> > per-user basis (~/.resource-file) and allow that default to be
> > overridden on a per-execution basis.
>
> Existing tools need a file view. (please don't use "atomic")

Existing programmes *do not* need to see albods as atomic. My custom
backup programme is better off seeing an albod as a directory.

> BTW, existing tools are already compiled and linked.

Yep. My custom tools are statically linked.

> Ugh. I don't think /bin/mv should be reading config files in your
> home directory.

mv(1) should not require a directory-based adbod to be automagically
tarred and untarred.

Now, I realise that some people may say that mv(1) won't move
directories across filesystems. True. We should fix mv(1) to
effectively do <cp -a; rm -rf> (with error checking).

Don't be confused by thinking mv(1) is an interface to rename(2),
because it isn't. mv(1) can move files across filesystems, and file
and directories within a filesystem. So mv(1) is lacking in symmetry.
Let's fix that.

> >>> Other (optional) behaviour can be added on top of this.
> >>
> >> This can not be. If you really think that random user-space software
> >> developers will agree on a way to use directories as documents...
> >
> > That doesn't matter. If GUI developers can't agree to use a common
> > library, it does not follow that the functionality should be pushed
> > into the kernel/libc.
>
> Hey, maybe Microsoft is right about us...
>
> I said the above mostly because I think your "userspace" argument
> is really a disguised "go away and die" argument. I believe you are
> fully aware that a userspace implementation can not happen, and you
> are counting on that fact.

I take offence at that. You are insinuating that I'm being two-faced.
Whether I'm write or wrong on the technical issues, I say what I
think.

Thus, when I propose a user space solution, I do believe it can
work. It's not a trick.

> If you don't like multi-forked files, just say so.

I've got no problems with directory-based albods where the data is
spread in multiple files. In my environment I often have to deal with
them, and they make a lot of sense for certain types of data.

> >> It isn't enough to have a few GUI apps. Users will be confused by
> >> the inconsistent treatment.
> >
> > No they won't. Each user can edit ~/.albodrc. Generic luser has:
>
> Hmmm, anybody that doesn't want to screw with tar is a luser?
> You have too much time to waste.

I didn't say that at all. Go read what I said. My scheme allows all
kinds of users to get the kind of view they prefer. Some will prefer a
cooked mode, others will prefer a raw mode.
I don't see what tar (specifically) has to do with it.

> > The point is everybody can be catered to with this scheme. Hacking the
> > kernel/libc *prevents* a *legitimate* mode of operation. That is
> > simply unacceptable.
>
> That is simply not true.

Really? So tell me how my (possibly statically linked, but not
necessarily) ls, mv, cp (whatever) binaries can show me that a
directory-based albod is a directory, if you put in a kernel or libc
hack?

> BTW, devfs *prevents* a *legitimate* mode of operation. Once people
> start to use it, developers will require it, and then nobody will
> be able to have a stable /dev with reliable security. :-/

Firstly, exactly how will developers require it?
Secondly, where are the stability/security problems with devfs+devfsd?

> What is your real concern?

Again, you're implying I have a hidden agenda. I don't, and it's not
appropriate to make such insinuations.

> >>>>> Command-line users who want to see albods as atomic can use some
> >>>>> special tools, or perhaps switches to existing tools.
> >>
> >> Command-line users who want to see the parts can use special tools.
> >> $ albod -x 80A8C452 ~/foo.doc > a.png
> >
> > NO! I want *all* my existing tools to be unaffected. I don't want the
> > semantics changed. But I'm quite happy for another user on the machine
> > to use cooked mode.
>
> By opposing this, you get the alternative:
>
> $ doc-fmt -x 80A8C452 ~/foo.doc > a.png
> $ id-fmt -x 80A8C452 ~/foo.id > b.png
> $ kde-fmt -x 80A8C452 ~/foo.kde > c.png
> $ xml-fmt -x 80A8C452 ~/foo.xml > d.png
> ...
>
> See? You will be even less able to use your command-line tools.

No, I don't see. I don't see the connection between this and the
proposal I made. Could you explain what you mean?

> >> I'd say you are afraid of change. When the topic isn't devfs... :-/
> >
> > I'm afraid of having different semantics shoved down my throat.
>
> Many are afraid that devfs will become a required feature.
> While devfs is currently an option, you might shove it down
> people's throats at some point. (maybe a driver will need it)

If a driver author requires devfs, that's not the same as me shoving
it down people's throats.

> > With kernel space albods, I don't get the choice of having all my
> > tools work in raw mode. This is why I'm fundamentally opposed to
> > them. The key point is that I support the availability of new
> > semantics but I oppose the restriction of existing ones.
>
> No, this is totally backwards. If you don't want to use multi-forked
> files, then just don't use them. You are "effectively trying to
> prevent me [...] from having the full range of choice".

You're not referring to the scheme I outlined. My scheme allows you to
alter the view a user has of an albod, on a per-user basis. So my
scheme allows you to view albods as atomic, and allows me to view them
as directories (when using system utilities or a file browser).

Exactly what does my scheme prevent you from doing?

> >> What use are the raw components? They won't be in any file format
> >> that you would normally use. They may be headerless raw image data,
> >> binary markup language, binary data structures, etc.
> >
> > That's not true. They may have gif files, for example. And the headers
> > are somewhere in the albod directory.
>
> Oh, you wish. Office apps tend to have their own internal formats.

Sure. And some will be more script-friendly than others. I would hope
that KWord (or whatever) doesn't make it's file format deliberately
inscrutable.

> > ??? I'm talking about system utilities, ls, sed, awk, perl, tr plus a
> > whole bunch of my own custom utilities.
>
> Few people use "sed" on raw binary data.

There's no reason to assume that a word processor document has to
store the text component in binary format. So my argument holds.

Regards,

Richard....

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/