Re: Implementing Meta File information in Linux (resent after bounce

L. Adrian Griffis (adrian@idir.net)
Mon, 07 Sep 1998 15:57:37 -0500


This is a multi-part message in MIME format.

--------------3AACC62313762E172435A2C
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I sent this before, but it bounced, complaining about a signal 15.
The bounce message and the fact that I haven't been flamed for
daring to make the suggestion both make me think I need to send
this again. 8-)

Hopefully, the bounce message will help someone figure out what
went wrong the first time.

---
L. Adrian Griffis - KE6CSX - adrian@idir.net

--------------3AACC62313762E172435A2C Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Content-Disposition: inline

Received: by cypress for adrian (with Cubic Circle's cucipop (v1.21 1997/08/10) Mon Sep 7 14:29:51 1998) X-From_: MAILER-DAEMON@cypress.idir.net Fri Sep 4 00:53:43 1998 Return-Path: <> Received: from vger.rutgers.edu (root@vger.rutgers.edu [128.6.190.2]) by cypress.idir.net (8.8.5/8.8.4) with ESMTP id AAA27537 for <adrian@idir.net>; Fri, 4 Sep 1998 00:53:42 -0500 Received: by vger.rutgers.edu id <154175-20492>; Thu, 3 Sep 1998 23:10:01 -0400 To: adrian@idir.net From: The Post Office <postmaster@vger.rutgers.edu> Sender: mailer-daemon@vger.rutgers.edu Subject: Delivery problems with your mail Cc: The Post Office <postoffice@vger.rutgers.edu> MIME-Version: 1.0 Content-Type: multipart/report; report-type=delivery-status; boundary="S.pvpc7154175=_/vger.rutgers.edu" Message-Id: <19980904033813Z154175-20492+22@vger.rutgers.edu> Date: Thu, 3 Sep 1998 23:10:01 -0400

--S.pvpc7154175=_/vger.rutgers.edu Content-Type: text/plain

A copy of your message is being returned to you due to difficulties encountered while attempting to deliver your mail.

The following errors occurred during message delivery processing:

<local linux-kernel@vger.rutgers.edu "|/usr/local/etc/majordomo/wrapper resend -p bulk -M 10000 -l linux-kernel -f linux-kernel -h vger.rutgers.edu -s linux-kernel-outgoing-dig" 0>: [signal 15]

--S.pvpc7154175=_/vger.rutgers.edu Content-Type: message/delivery-status

Reporting-MTA: dns; vger.rutgers.edu Arrival-Date: Thu, 3 Sep 1998 18:52:45 -0400

Original-Recipient: rfc822;linux-kernel@vger.rutgers.edu Final-Recipient: X-LOCAL;"|/usr/local/etc/majordomo/wrapper resend -p bulk -M 10000 -l linux-kernel -f linux-kernel -h vger.rutgers.edu -s linux-kernel-outgoing-dig" Action: failed Status: 5.3.0 (subprogram non-zero termination code) Diagnostic-Code: x-local; 500 ([signal 15]) Last-Attempt-Date: Thu, 3 Sep 1998 19:29:15 -0400

--S.pvpc7154175=_/vger.rutgers.edu Content-Type: message/rfc822

Received: from port088.tnt1.kc.idir.net ([209.172.192.187]:2460 "EHLO typhoon.nerds.org" ident: "root") by vger.rutgers.edu with ESMTP id <154499-28979>; Thu, 3 Sep 1998 18:52:45 -0400 Received: from typhoon (adrian@localhost [127.0.0.1]) by typhoon.nerds.org (8.8.7/8.8.7) with SMTP id UAA05053; Thu, 3 Sep 1998 20:17:03 -0500 Sender: adrian@typhoon.nerds.org Message-ID: <35EF3F8E.542C7EE4@idir.net> Date: Thu, 03 Sep 1998 20:17:02 -0500 From: "L. Adrian Griffis" <adrian@idir.net> Organization: Just Me X-Mailer: Mozilla 3.01Gold (X11; U; Linux 2.0.30 i486) MIME-Version: 1.0 To: linux-kernel@vger.rutgers.edu Subject: Re: Implementing Meta File information in Linux References: <19980902110007Z154141-17700+5482@vger.rutgers.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit

Richard Gooch <rgooch@atnf.csiro.au> wrote: > Really, ACLs and icons are very different classes of meta-data. > [...] > The best approach is to have all meta-data inside a plain file. In the > case of datafiles which have header information associated with the > raw data, it's not too hard to do this. Where you can't put meta-data > in with the file data (such as icons for an executable), make your > meta-file a directory with the components being files within it. The > specialised tools that can deal with this meta-data will know that a > directory with an "exe" file is really a programme that can be run, > which might have a pretty icon. "Dumb" tools can still be used to > manipulate the meta-data components and sysadmins won't require deep > knowledge of the meta-data format (it will be obvious from the > filenames). Trust me: in the long run everyone will thank you for > doing it this way :-)

I have another idea: Suppose we take the inode substructures for keeping track of file length and blocks used and created a duplicate of these structures. This way we would have a meta data area tracked in the inode along with the data area. The meta data area could function like the data area for a directory, i.e. you can not simply write to it; Instead you can only manipulate the data area via system calls that are specific to it. Also, the data area could be organized into variable length logical entries, in the same manner that directories are. Each entry could be organized into a tag and a value. Some characteristic of the tag (e.g. the first byte or the high order bit) could determine if it is a kernel tag or a user tag. The entire meta data area could be read with an open_meta() call, and parsed something like the way we parse a directory.

This way, meta data would be associated with the inode, so it would not be subject to confusion if the file is linked in multiple directories, each with it's own ".metadata" file. The kernel could have some easy way to determine its level of interrest in a tag. This approach is flexable enough that we would not have to make fundimental changes to the inode structure to add additional meta data functionality, we would only go through that pain once. A meta-aware version of tar could save all the meta without having to understand what any of it means. It could restore the metadata with an insert_meta_entry() call, and the kernel would be free to reject any request that it doesn't like, again without tar having to understand what the tags are. The kernel could return with the error EDONTDOTHAT for tags that are never altered by the user, and tar could know not to complain and not to try to set that tag for any later files. You could save and restore individual files, without having to go through some complex locking, reading, and updating process on the ".metadata" file that holds info for all files in a directory.

User tags could be assigned in the same way we assign well known ports under IP, without the kernel caring what they mean. The meta data for a file that is still open, but has been unlinked, would still be accessable through the "/proc" subdirectory for the process that has it open.

Sooner of later, I think it will be handy to be able to attach meta data to the inode for a file, and not to the directory (even in a logical sense). No matter how you look at it, attaching more kinds of data to the inode will require changing the inode structure, and this means creating a new filesystem. Shouldn't we be coming up with a flexible enough scheme to keep us from having to jump filesystems any more than necessary? The VFS hooks to support what I propose would also give us a means to fake some of the more complex features of non-unix, filesystems like NTFS, SMBFS, and Novel, even if we did not know about them before freezing the inode structure layout. A loadable module could register its ownership of a "kernel" tag that the base kernel didn't understand. It could also register itself into a list of modules to be called before a "EPERM" error is indicated to the user. People who didn't want the kernel bloat associated with a complex ACL scheme could leave it out. people who wanted to experiment with a really bizarre tag could load and unload it without rebooting, and without a creating a new type of filesystem for each iteration of the feature being designed.

Just an idea....

---
L. Adrian Griffis - KE6CSX - adrian@idir.net
--S.pvpc7154175=_/vger.rutgers.edu--

--------------3AACC62313762E172435A2C--

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/faq.html