Re: 'disposable' dirty pages [was: Out Of Memory in v. 2.1]

Jeremy Fitzhardinge (jsgf@sirius.com)
Thu, 08 Oct 1998 12:01:43 -0700 (PDT)


On 08-Oct-98 Jamie Lokier wrote:
> On Wed, Oct 07, 1998 at 12:03:48PM +0200, MOLNAR Ingo wrote:
>> if it's the libc malloc cache itself, then there need to be no action,
>> other than to remove that given area from the malloc cache. Due to the
>> asynchron behavior, more complex uses might be tricky.
>
> This should be possible without a fault handler and minimal libc support.
>
> When free() creates a large enough contiguous free region to be worth a
> syscall, call mmap(..., MAP_ANON | MAP_PRIVATE | MAP_DISCARDABLE, ...)
> to make an "optional zero-map" of the region.

Why the MAP_DISCARDABLE? What does it give you that just mapping MAP_ANON over
the memory doesn't already give you? If the kernel can asynchronously take the
pages away from the process at any time, they're effectively unusable, so you
may as well just remap them as ZFOD pages as soon as you're done with them.
What happens if you write a MAP_DISCARDABLE page? Does it become
non-discardable? If not, how can you use the page again later without the
kernel stealing it from you?

If you want disposable pages, you need to design a system which is more useful
to a general class of programs. Dealing with malloc is easy, it doesn't need
any more mechanisms, but consider a garbage collecting allocator:

- allocation is (generally) very cheap
-- basic trade-off is allocate more heap or try to reclaim space in existing
heap?
- garbage collection is somewhat expensive
-- but you only need to do it if you're running out of memory

A typical GC has no idea whether the system is running out of memory (that's
physical memory - garbage collecting is much faster than paging), which it
needs to know to decide whether to expand its heap or to do a collection cycle.
It can probably find a reasonable amount of stuff to return to the OS if it
does a GC, but it wants to delay doing a GC as long as possible.

The problem with MAP_DISCARDABLE is that it assumes that you already know what
memory is expendable from the outset, but if you know that, you can free it
anyway. The normal reason malloc doesn't free pages is that it wants to avoid
making syscalls, but MAP_DISCARDABLE doesn't help there.

An async notification of low memory from the kernel is more useful. It lets
the GC know that it really should collect memory. It also helps a malloc()
implementation to know whether it should start unmapping the free parts of its
heap. The downside is that its pretty ugly. And you could see it as a nice
convenient mechanism for a malicious program to know that its doing its job
right...

J

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/