Re: Filesize limitation

Chel van Gennip (linux@vangennip.nl)
Sat, 8 Nov 1997 14:05:55 +0000 (WET)


"Albert D. Cahalan" <acahalan@cs.uml.edu> wrote:
>Chel van Gennip writes:
>
>> I think there is a real need for files > 2GB.
>
>Linux will support such files when the _is_ a real need.
>Over the years, I've seen less than a dozen people complain.
>As for myself, 2 GB would be an acceptable hard disk limit.
>

I think the time is about now. Unit capacity doubles every 2 years,
the size for a 3.5" disk is now 20GB.

2 GB is sufficient for most purposes. Common use for bigger sizes
will be:
- mastering DVD disks
- recording and editting video
...
I could use larger files to store backups from other machines in
a backupserver.

I think there is a real need for this in 2.2 kernels.

>> A real problem is in the ext2 filesystem:
>...
>> How difficult would it be to upgrade this to a ext3 filesystem
>> with 32 bits reserved for gid and uid, and 64 bits for length
>> and block count?
>
>There is enough space in the existing inode. You did not look
>at osd2, the union for operating system dependent features.
>Linux and Masix have mostly reserved space, but look at what
>is already filled in for the hurd:
>
>__u8 h_i_frag; /* Fragment number */
>__u8 h_i_fsize; /* Fragment size */
>__u16 h_i_mode_high;
>__u16 h_i_uid_high;
>__u16 h_i_gid_high;
>__u32 h_i_author;
>
>Without trouble, we have 32-bit UID and GID support.
>Now look at h_i_frag and h_i_fsize. Those exist to support
>a feature that was never implemented in ext2. Since the feature
>does more harm than good, the osd2 union (for Linux) can become:
>
>__u16 l_i_size_high; /* high 16 bits of 48-bit file size */
>__u16 l_i_pad1; /* reserved for Hurd permission bits */
>__u16 l_i_uid_high; /* high 16 bits of 32-bit UID */
>__u16 l_i_gid_high; /* high 16 bits of 32-bit GID */
>__u32 l_i_pad2; /* reserved for Hurd author ID */
>
>The block number limit is not a serious problem. If anybody hits
>that limit, they will get better performance from 64 kB blocks.
>That would allow for a 48-bit (256 TB) file & filesystem.
>Such a system is likely to have 4 TB of RAM, which is near the
>theoretical hardware limit of current Alpha CPUs. I think we have
>at least 10 years before such a system ships. Good enough for now!
>
>BTW, these unions could be merged in with the rest of the struct.
>It would be dangerous to actually make the unions members different,
>so there is not much reason to have the ugly code.

I think this is real good! We could have a filesystem compattible
with the existing ext2.

Some people wanted 128 bit filsystems. I think that is overkill.
At this moment 2GB is allmost enough. If the demand rises at the
same speed as technology (2x in 2 years) 16 extra bits will give
32 extra years. For me that will do!

What has to be done?
- change asm/stat.h to have a full length in__unused4/__unused5
- stat() will give 0xffffffff in st_size if the size exeeds 32 bits
- ext2fs handling should be modified to handle the new inode layout.
- mimimal support is vailable through the modified stat() and llseek().
- fsck.ext2 should be modified to handle the new length extension.
- mkfs.ext2 should clear these fields, but I think this is allready OK.

Chel.