Re: Endless overcommit memory thread.

From: Olaf Weber (olaf@infovore.xs4all.nl)
Date: Sun Mar 26 2000 - 14:03:28 EST


Riley Williams writes:

>> Can someone please explain to me how and why an app would allocate
>> a large amount of memory and then not use it? Please give specific
>> examples since I can't fathom the reason.

> OK, here's one application that COMMONLY requires large amounts
> of memory, but under certain rare situations needs only a small
> amount of memory: Capturing video direct from TV to disk when one
> can't store it in the format it arrives in, for whatever reason.

As point out, the common case is that the memory is actually used.
Moreover: the requested memory must be available the moment it turns
out to be required, so the pages must be locked in physical memory.

The application cannot afford to miss its time-slice, so must run at a
higher than normal priority (preferably a real-time priority).

And you want to limit the additional load on the machine, again to
ensure that it doesn't impose overheads that would kill the RT
performance.

In all, this is an atypical example.

A common example are applications using sparse arrays. It is often
cheaper to waste some memory (even the pages that are used need not be
densely used) than to incur the computational overhead of techniques
that turn a sparce array into a dense one.

Another example is an application that uses a potentially large number
of objects which live in an "arena". Using realloc to enlarge an
allocated arena means the arena can move: the price you pay for that
is that you have to address objects in the arena by their offset from
the base pointer, instead of using addresses directly. This kind of
overhead can be significant. So you allocate an arena that will
always be large enough, even if it is over-large for most cases.

And there is the traditional fork/exec when the forking application is
large (say, emacs or netscape) and the forked application is small
(say, a shell). clone/exec is an alternative on linux.

-- 
Olaf Weber

Do not meddle in the affairs of sysadmins, for they are quick to anger and have no need for subtlety.

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Mar 31 2000 - 21:00:17 EST