Serious bug in mmap; ZMAGIC design flaw

Eric Gisin (ericg@unixg.ubc.ca)
Wed, 26 Jul 1995 06:49:27 GMT


The current version of mmap allows file offsets that are
a multiple of 1024 (ext2 block size) but not a multiple of 4096
(the i386 page size). Before shared mmaps in 1.3, this was a minor
problem for ZMAGIC a.out mapping on some file systems (look
at the mmap hacks in fs/msdos and fs/umsdos).

With shared mmaps in 1.3, it becomes a serious problem.
If you mmap a file with offset 2048, you get the address of
a page for bytes 2048:6144-1. If you then mmap the same file
with an offset of 0, mmap cannot return an address that
shares the same page as the first mmap, because the i386 pagesize
is 4096. The two mmaps both succeed, but they return
two distinct pages and shared modifications are not seen
between the two mmap'd regions.

The only fix to this is to require the file offset to be
a multiple of the page size. But this would completely break
the execution of ZMAGIC files, which rely on mmap succeeding
with a file offset of 1024. Since exec uses private, not shared,
file mapping, we could make an exception for private mmaps.
ELF binaries always map the file on page boundries.

My previous mmap bug report contains a test program that
also demonstrates this bug. Use "a.out 2048".

Eric Gisin, ericg@unixg.ubc.ca