Date: Wed, 9 Feb 2000 21:15:14 +0000 (GMT)
From: Riley Williams <rhw@MemAlpha.CX>
>> Linus made it clear some time ago that ANY time a file system
>> changed in ways that meant that it could no longer access older
>> formats, it would be given a new name. That's the reason why ext
>> became ext2 in the first place, and also why ext3 has its own
>> name. As a result, that argument is a non-starter.
I note that you ignore the claim that "ext became ext2" and
concentrate on ext3 here...
ext to ext2 was a real format change. There was no compatibility at all
between ext and ext2 tools, in either direction. The same can't be said
of ext3.
> Once you unmount an ext3 filesystem cleanly, the journal bit is
> cleared, and a standard ext2 filesystem code and standard ext2
> utilities will work just fine with the filesystem.
First, your use of the word "cleanly" invalidates your claim that it's
backwards compatible since the latter claim is meaningless unless it's
true with unclean unmounts.
Umm... hardly. It's never safe to mount a filesystem that's been
uncleanly unmounted. And e2fsck will eventually be able to deal with an
uncleanly unmounted ext3 partition. Support for that hasn't been added
yet, because the ext3 journal format hasn't been finalized yet.
But that's the same issue as with e2compr. E2fsck doesn't handle
e2compr enabled filesystems, either. But it will, once e2compr and its
format are stablized, and set into stone. It's the same issue with ext3
journaling.... and with sparse superblocks. You need new kernels and
new tools to handle sparse superblocks, but the new kernels and new
tools will happily support the old filesystem.
Secondly, I had occasions where even with a clean unmount, the journal
bit was NOT cleared, so the claim isn't necessarily true anyway.
There is a compatible HAS_JOURNAL flag, and an incompatible
NEEDS_RECOVERY flag. The HAS_JOURNAL flag is in the compatibility
bitmap, so old kernels will happily mount filesystem with that bit set.
The NEEDS_RECOVERY flag means that the journal has to be run before the
filesystem is sane. Kernels and e2fsprogs that don't know about that
flag will refuse to touch a filesystem with that flag set. In the
future, though, kernels and e2fsprogs *will* know how to run the
journal. (I'll add that as soon as the journal format gets stablized.)
> What I was talking about is the backwards incompatible changes.
> For example, the change between e2compr 0.3 and e2compr 0.4.
> While that sort of change is still a possibility, IMHO it
> shouldn't go into the kernel, since we *don't* want to encourage
> widespread adoption of something that we might need to change
> later.
That's presumably the reason why there was massive file system
breakage early in the 2.3 kernel series when Linus made changes that
stopped most of the file systems from compiling?
Linus made API changes that stopped things from compiling. If Linus
makes API changes that break the world, eventually we restablize the
kernel series, and people can go to a newer kernel and they're all set.
But with a filesystem format change, if people start using a filesystem
format that you know you're going to break later on, going to a new
kernel doesn't help them. But if a lot of people were using the
incompatible 0.3 e2compr filesystem, we're forced to keep the
compatibility code in place for a long time.
That's why it's a **good** thing that e2compr didn't get folded into the
Linux kernel tree before the 0.3 to 0.4 format change. And, if you want
to make other format changes, at this point, you'll thank your lucky
stars that 0.4 didn't make it into the kernel, either....
The only 'backwards compatibility' code that I'm interested in seeing
in the kernel would be code that decompresses depreciated compression
algorithms and forcibly labels any files using those algorithms as
read-only. As I see it, that's the ONLY sensible way to deal with such
algorithms - and to prevent them taking up unnecessary kernel memory,
force them to be modules rather than hard compiled in.
I wouldn't even do that much. Take the code out of the kernel entirely,
and then provide a user-mode utility that goes through the filesystem,
and converts the files compressed with a depcreated algorithm to either
being uncompressed or compressed with a better format. The goal here is
to keep the kernel as long-term maintainable as possible.
Otherwise, the depcrecated algorithms will be in the kernel tree for a
*long* time. That's why it's good not to let stuff like this into the
main kernel tree until you have a better idea what you want to take out.
The fewer people you have using the deprecated filesystem format, the
more easily you can get rid of that code bloating the kernel.
> 3) There is space to support at most 32 algorithms. Do you
> really need to chew up that space with 9 different GZIP variants
> (for gzip -1 through gzip -9) and 3 different Lempel-Ziv
> variants?
Based on what I've looked through so far, at least one of the L-Z
variants only exists in the documentation as it has already been
depreciated and withdrawn. I can also see reason to reduce the number
of valid gzip levels to at most 3, along the lines of the following
entries in the Config.in file:
choose ' GZIP compression strategy' \
"SPEED 'Favour speed over compression' \
NORMAL 'Speed equal to compression' \
SPACE 'Favour compression over speed'" NORMAL
The code would then decide to use -1 or -5 or -9 depending on which of
those three was chosen. The other levels are effectively meaningless
in this context, in my opinion.
Well, the e2compr code's header file defines 24 out of the 32
algorithms, with 3 of them being the L-Z variants, and 9 of them being
gzip variants. If they're really gone, remove them from the header
file. The header file needs to be your allocation table of record of
what compression algorithms are supported and which ones aren't.
What I suggest that you do is that you figure out where on the
speed/compression performance axis the 20 odd compression algorithms
supported by e2compr fall, and then decide how many algorithms you
*really* need to support. That list might be modified somewhat by
patent issues, but if a compression algorithm is 5% faster, but has a
patent covering its use in some countries, is it really worth including
that algorithm just for the 5% speed improvement? Some tradeoffs need
to be made here, I think. What's the point of supporting 24 different
algorithms if people only use 3 or 4 of them?
Whilst the level isn't needed to decompress, I'm not so sure that the
same is true when an application just appends to an existing file
without reading it. However, I will be looking into that in the near
future.
Each compressed cluster is independent (it has to be, so that random
access is efficient). As such, it's perfectly safe to have different
parts of the file compressed at different gzip levels.
- Ted
P.S. If you really think we need to support that many compression
algorithms, why not use 3 bits in each inode to represent which
compresion algorithm to use, and then use those 3 bits to index into a
table in the superblock that has a 32-bit compression algorithm
identifier.
That way you can support a huge number of compression algorithms, but
each filesystem only can support at most 8 at a time --- which I suspect
should be plenty. But that way, if each person wants to support their
own pet compression algorithm (that they use and no one else does),
there's space to allow them to do so. (I observe that deciding which
compression algorithms to support and desupport is probably more of a
political issue than a technical issue. :-)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Tue Feb 15 2000 - 21:00:16 EST