Ideas for memory management hackers (2)

Hans-Christoph Rohland (
16 Nov 1997 13:53:34 +0100


(I discussed some of these items with David Miller at Linux Kongress,
Germany. Dave you remember?)

I think the virtual memory support in linux has the following
weaknesses irt. applications which use loads of virtual memory in
parallel processes:

- Support for shared pages:
- No aging (?)
- Bad handling of dirty pages. They have to be written to disk by
every process which changed the page before releasing the page.

- VM areas are limited to 128MB. This leads to many swap-files. (I am
not quite shure about further limitation. Does it mean that bigger
mmaps are not possible? The swapfile limitation is definitely in the

- Shared anonymous mappings are not supported.

- SysV-IPC is difficult to tune in high memory situations because you
have to decide between many or big segments before compiling the
kernel. Also it is handled in the kernel specially.

What I am thinking of is

1) Good support for swapping of shared pages

2) Bigger VM areas

3) A generic mechanism for anonymous pages. On top of this could be
build shared anonymous mappings, SysV-SHM and
swapfilesystems/named files in swap.

4) implementation of the 'madvise' call, especially the MADV_DONTNEED

Some comments:

With swapfilesystemes or named files in swap you increase the power
of mmap for shared memory. You can open a file in swapspace and by
mmapping areas into your address space you have a very flexible
mechanism for sharing memory between processes. With normal named
files you get performance problems due to syncing the file during
unmapping and also because swapping to filesystem is slower than to
swap devices.

The need for MADV_DONTNEED is a similar problem: no memory manager can
know that I really don't need a part of memory any more. It will
always try to save the memory in backing store.
But especially transactional programs process some memory and when the
transaction finishes there is no need to get this content again. It
will be overwritten by the next transaction. So the program can
advise the OS to simply discard the page instead of saving the page to
the swapfile. This can increase the system performance a lot under low
physical memory.