Re: Overcommitable memory??

From: John Ripley (john@empeg.com)
Date: Sat Mar 18 2000 - 10:27:40 EST


Andreas Bombe wrote:
>
> On Thu, Mar 16, 2000 at 11:22:17AM +0000, Paul Jakma wrote:
> > On Thu, 16 Mar 2000, James Sutherland wrote:
> > > On Wed, 15 Mar 2000, Paul Jakma wrote:
> > > That doesn't happen. malloc() ALLOCATES the memory to the process. It is
> > > *NOT* overcommitted. It may be backed by swapspace rather than physical
> > > memory, but that block of memory *IS* available to the process.
> > so malloc() isn't overcommited? malloc()'ed memory is guaranteed to be
> > available - ie the memory is reserved and accounted for at malloc()
> > time?
> > (i always thought malloc()'s were noted by the kernel ie in the page
> > table for the process, but that the kernel didn't actually allocate a
> > real page until the process tried to access that page - ie malloc() is
> > overcommited)

> Looking at the sources of do_brk(), it faults in the pages (thereby
> assuring they will exist for the process) only if the process' VM is
> locked (mlockall()). Also, it returns with ENOMEM only[1] if
> vm_enough_memory() says there isn't enough mem, but this function only
> examines a snapshot of mem usage at the time of the call (and is
> always successful if overcommit is set to on).

Even mlockall() doesn't get you out of overcommit because the
make_pages_present() call in do_mmap will fault all pages in and SIGBUS
your process rather than return -ENOMEM.

There are quite a number of problems with mlockall:

- If there's not enough phsyical memory to satisfy the current process,
you die. No ENOMEM.

- There's a nasty and quite arbitrary hard limit on half of physical
memory.

- Read-only mapped file pages are still counted towards the cache total
(page_cache_size) even though they're not allowed to be paged out. So in
calculations for free memory which take into account cache
(vm_enough_memory), there are more pages counted as freeable than there
really are.

The last point here I think is a bug, and not just choice of semantics,
unless I'm missing some subtlety burried deep in code.

Is there a document somewhere which specifies the exact semantics of
brk/mmap/mlock/nopage?

-- 
John Ripley, empeg Ltd.
http://www.empeg.com

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:24 EST