Re: DE-fragmenting the virtual memory map

Raymond Nijssen (raymond@woensel.ics.ele.tue.nl)
Wed, 8 Sep 1999 23:20:20 -0700 (PDT)


::: "BL" == Benjamin LaHaise <kernel@kvack.org> writes:

> On Tue, 7 Sep 1999, Raymond Nijssen wrote:
>> The limitations of this layout are:
>> 1. the effective heap size of about only 0.8 GB. This is insufficient for
>> large processes. Even on WNT you get effectively about twice that,
>> and it is not enough either. Solaris/SPARC provides 3.75GB user vm.

> You're missing a very important point: the normal implementation of malloc
> under linux makes use of mmap() to allocate blocks of memory. Personally,

No, that is not the same. mmap()ing only kicks in when large blocks are
allocated. I worked with Doug on implementing this feature.

> I consider brk() an obsolete interface because it just doesn't fit with
> the normal allocation/deallocation patterns of daemons and applications --
> pages are not returned to the system that are otherwise unused, and end up
> needing to be swapped out.

If you were to do what you propose you would need to allocate one page for
every allocated chunk. In fact most requests to malloc() are just a few bytes
large, say 64 bytes, while a page is 4k. The likelyhood that all chunks in
the same page are all freed is very small.

I believe the malloc() package is doing exactly the right thing by using
mmap() for large requests while packing smaller requests in a pool that it
administers. Whether it obtains that pool from the kernel through sbrk() or
mmap() makes no real difference on Linux, Solaris and a couple of other modern
unices, but remember that the package is also supposed to work on other
platforms on which that does make a difference.

> Note that by default, the space available for
> mmap()ing is just under 2GB on ia32. That's reasonable for a 32 bit
> architechure.

This is incorrect. You can easily mmap() a total of about 2.75 GB on
linux/x86. BTW on Solaris/SPARC you get 3.75GB in a 32 bit environment. On
the verge of the 64-bit era you're going to need all you can get.

-Raymond

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/