(reiserfs) Re: RE: (reiserfs) File conglomerations

Hans Reiser (reiser@ceic.com)
Thu, 1 Jul 1999 12:29:32 +0000 (/etc/localtime)


Yes, storage layers should be separated from semantics. The code sort
of does this, what it lacks is an kernel API. Some day....

Hans

Tan Pong Heng writes:
> What make reiserfs reiserfs at the moment is the way it organises the fs
> and map that into the physical media. It may have some specific
> functionality
> that are not available from other fs - either due to the way it does
> its works or in addition to what the standard fs provides.
>
> What I propose is that the fs functionality can be analyzed and grouped
> into various groups/layers. The way the fs is organised on the physical
> media is one group/layer. The aggregation functionality belong to another
> group/layer and is basically orthogonal to the first group/layer. If you
> look at it that way, there is no reason why they can not be splited and
> implemented separately in such a way that they can be mixed and matched.
> That way, you can assure that each can be developered separately in
> the most efficient way. Also, it would ensure that nobody will be wasting
> efforts into implementing the same aggregation functionality in other fs
> if it is proved effective.
>
> I think it is too early to commit whether the aggregation functionality is
> useful/effective or not. But, at least reiserfs has been proven in some way
> that it is efficient/effective for some specific purpose. As such, why
> not keep these two groups of functionality reasonably separated so that
> they can be developed in the way that are most suitable for them?
>
> I believe the UNIX concept of modularization is the right way for software
> development. It basically enable reuse of codes.
>
> Packaging functionality together and pushing them out together is Microsoft
> way of forcing functionality to users to ensure the survivability as a
> whole.
> While it may be a viable approach for commercial software - it should not
> be the way to go for open source development.
>
> ----- Original Message -----
> From: Hans Reiser <reiser@ceic.com>
> To: Tan Pong Heng <pongheng@starnet.gov.sg>
> Cc: <linux-kernel@vger.rutgers.edu>; <reiserfs@devlinux.com>
> Sent: Wednesday, June 30, 1999 8:42 PM
> Subject: (reiserfs) RE: (reiserfs) File conglomerations
>
>
> >
> > None of the functionality of reiserfs should be specific to reiserfs....
> >
> > I completely reject your argument that I must stay in the center of the
> > herd, and only implement functionality that others have done in other
> > file systems. It is time for the herd to move, it can follow me if it
> > wants to.
> >
> > Hans
> >
> > Tan Pong Heng writes:
> > > After listening for so long, I came to the following "conclusions" - 2
> of
> > > them:
> > > 1) Whether it should be done?
> > > 2) How it should be done?
> > >
> > > The first question can only be answered after you are clear why you
> want to
> > > do it. So far, I have seen only one "justification" - programmer
> > > implementing
> > > structure store themselves. For this, we have to understand why
> structure
> > > stores were introduced in the first place. It seems rather clear that
> in
> > > most
> > > cases, structure stores could be replaced with directory trees - except
> for
> > > on aspect - you still want to treat the structure store as one entity
> and
> > > handle
> > > as such. You want to be able to identify, copy, delete, and operate on
> a
> > > structure
> > > store as a file. The file aggregation extension proposed would meet
> most of
> > > these requirements. But, there is still two other important aspects -
> > > portability and
> > > the ability to reverse the aggregation operation. The application
> programmer
> > > implement their own structure store so that they can ensure that the
> file is
> > > portable
> > > across operating systems. Implementing this as a file system extensions
> > > specific
> > > to Linux does not meet this requirement. Another problem is that, file
> > > aggregation
> > > can be done easily, but the reverse is rather hard to do correctly. For
> > > example, if
> > > you use tar to aggregate, does that mean that when a tar file is copied
> in,
> > > you
> > > untar it automatically? If you don't, you can not preserve the
> semantics.
> > > As such, until a good answer for these issues surface, there is no real
> > > reason
> > > to rush into this.
> > >
> > > Even if you want to do it, whether within a file system such as RiserFS
> is
> > > the
> > > right place to implement it is another important question. Please note
> that
> > > these
> > > functionality should not be specific to any file system - why can't I
> have
> > > it on top
> > > of NFS, E2FS, VFAT, etc? The current "design" of the UNIX FS is that
> there
> > > is
> > > a VFS layer and the underlying FS layer. The underlying FS layer
> implement
> > > the layout of FS on the physical devices - and it is their jobs to do
> this
> > > well.
> > > In that aspect, RiserFS has proven to have done it reasonably well. The
> > > proposed
> > > functionality really has nothing to do with the physical layout. It
> should
> > > be implemented
> > > in the VFS layer. Or at least in a layer in between the existing two.
> In
> > > this way, the
> > > extension will be available to other FS too. Actually, I am considering
> to
> > > do it this way
> > > for the Crypto FS extension too. After all, why should it be limited to
> > > extending the NFS
> > > as in TCFS and CFS?
> > >
> > > Regards,
> > > Tan Pong Heng
> > >
> > > -----Original Message-----
> > > From: Lou Grinzo [mailto:lgrinzo@stny.lrun.com]
> > > Sent: Tuesday, June 29, 1999 11:18 PM
> > > To: linux-kernel@vger.rutgers.edu; reiserfs@devlinux.com
> > > Subject: (reiserfs) File conglomerations
> > >
> > > I must be insane to be still tilting at this windmill, but I have to
> > > give this one last shot.
> > >
> > > File conglomerations (albods, whatevers) is a very promising
> > > idea, but no matter how you approach it, it has an impact on
> > > file system semantics, which is a very serious change to any
> > > operating system. Such changes should never be made
> > > without a clear idea of what the goals are, and exactly how
> > > the system should look and work.
> > >
> > > The issues I personally would like to see addressed include:
> > >
> > > What benefits will this provide to the user working at a
> > > command line? What about a user running a GUI? Be
> > > specific, and think like a user, i.e. someone who is
> > > primarily interested in getting work done with the best
> > > combination of efficiency, ease of use, security, and
> > > stability possible.
> > >
> > > What are the benefits to a sys admin? Again, "be the
> > > admin" for the purpose of this answer.
> > >
> > > What benefits will this provide to programmers? Convince
> > > me as a Linux app. programmer that this is something I
> > > want to spend the time to learn how to use, and then
> > > actually support in my programs. What capabilities does
> > > this add to my toolkit, or what does it improve that I can
> > > already do?
> > >
> > > At each of the 3 key interfaces (CL, GUI, API), will file
> > > conglomerations always look like directory trees? Will
> > > they never look like directory trees? Or will they look like
> > > either, depending on semantic details? (E.g. use old file
> > > system calls to treat the cong. as a single file, but use a
> > > new call to treat it as a directory.) (IMO, the ability for the
> > > user to take an entire dir, tree of files and treat it as a
> > > single file (to move it to another system, for example)
> > > without having to resort to tar or any other special handling
> > > could be the biggest single end-user benefit of file cong.)
> > >
> > > Will file cong. be supported under all the FS's that Linux
> > > currently supports, and to an equal extent? If not, exactly
> > > how will this work with various FS's, as viewed by the three
> > > interfaces (CL, GUI, API)? (I'm not implying that a "no"
> > > answer to the first question in this paragraph means don't
> > > add cong.)
> > >
> > > What about changes to basic commands? Will any be needed
> > > to support dealing with a cong. as a single file vs. a directory?
> > > (It's very tempting to ignore this issue, and only discover
> > > afterwards that you've added a feature that has created a huge
> > > demand for "little" changes in dozens of commands. If it's
> > > deemed that this level and pervasiveness of change is
> > > acceptable, fine, but it's another detail that should be decided
> > > now and explicitly, not after the feature is rolled into major
> > > distributions and the issue is forced.)
> > >
> > > Will a CL user be able copy/move/delete/rename a cong. as
> > > a single entity without resorting to explicitly creating an
> > > archive of the cong.'s contents with tar or something similar?
> > >
> > > Will Windows NT cong. be treated like Linux cong., will they
> > > continue to be visible only as single binaries? (As Linux grows
> > > in mainstream usage, it will increasingly be used in mixed-mode
> > > environments on the desktop, making this a far more relevant
> > > issue.)
> > >
> > > Will this change entail tradeoffs in terms of system performance,
> > > usability, complexity, etc.? If so, what are they likely to be?
> > >
> > > Will this support be modular enough that a user or enterprise
> > > can choose not to use it and have zero impact on the system?
> > > In other words will it be "ignorable"?
> > >
> > > Will cong. have passwords? (Have to use the PW to mount it,
> > > and then it's a normal part of the FS.) Will cong. support
> > > compression? Encryption? (Yes, this is getting a bit blue-sky,
> > > but if it is decided that these features are definitely desirable,
> > > then it only makes sense to ensure that today's design is flexible
> > > enough to accommodate them when the time comes to add
> > > them.)
> > >
> > > Will security attributes be set for individual files in the cong.,
> > > for the cong. as a whole, or both--default settings at the
> > > cong. level, overridden by those of individual files?
> > >
> > > Will cong. support scripts/binaries that are stored in the cong.
> > > and automatically run when the cong. (not the FS that contains
> > > it) is mounted and unmounted?
> > > (There are some interesting possibilities here for software
> > > installation and de-installation, since this would provide most
> > > of the support for a very user friendly "software cartridge"
> > > architecture, something I've been working on the design of
> > > for a while.)
> > >
> > >
> > > Before anyone tries to lynch me, let me point out that in my
> > > experience in operating system and application design and
> > > programming, the two most valuable lessons I learned are
> > > that 1) too centralized and strict control over a software
> > > design is deadly, and 2) too little control is even worse than
> > > too much. The "right" amount of control is highly dependent
> > > on the nature of the project. Adding file cong. has many
> > > ramifications for the rest of the system, and requires a lot
> > > of up-front scrutiny, IMO, to add what's really needed, as
> > > well as to avoid problems in the future.
> > >
> > > I also want to say that I have a lot of respect for the Linux
> > > programmers and the distributed development model. I'm
> > > not in any way advocating a replacement for the current
> > > system. I'm proposing an extension of it to include a slightly
> > > more coordinated, and, hopefully, more complete and efficient
> > > analysis of requirements and the high-level design.
> > >
> > > As I've said before, if it appears that I can help the process
> > > by acting as an administrator to help spell out the issues and
> > > document the answers as you provide them, then I'll gladly
> > > volunteer and provide the web space.
> > >
> > >
> > >
> > > Lou
> > >
> > >
> >

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/