On Sun, 12 Mar 2000, Michael Bacarella wrote:
>
> > > I take it you run your sash mlock()ed with a special
> > > kernel patch to make sure procfs doesn't need to allocate
> > > memory on sash's behalf :)
> >
> > Unfortunatelly not.
> >
> > But if would be posible to somehow dividie the memory into overcommitable
> > and one which is not overcommitable it could help.
>
> Wouldn't this problem be avoided if the kernel DIDN'T overcommit memory?
>
> I mean, nobody tolerates their filesystem overcommitting blocks it
> doesn't have (or maybe they do and my reality is a myth). Why should
> it be tolerated for virtual memory?
>
> Are the benefits to saying "uh, sure. we only have 600 megs of VM, but uh,
> feel free to let your system commit 1 gig" worth it? What makes that
> behavior desirable?
>
> I'm not condemning, I'm just curious.
>
> -MB
It really isn't an option. Let's say you have a program that allocated
most of the memory in your machine. Then, the program needs to fork()
to do some simple command, the parent waits for the child to exit()
then continues. The nature of fork() is that all address space is
duplicated. The fork would not work unless the kernel over-committed.
The first line of defense is called COW, where the kernel only duplicates
pages if/when the child actually modifies them (Copy On Write).
You may think that this is a "special case". However, the kernel has
no prior information about what is being executed during the fork() so
it isn't special at all. In principle, the child could have accessed
all of the parent's memory and written to its copy.
The next problem is with sparse buffers, where a program allocates a
lot of data space because it might need it under certain conditions.
Many/most such programs never use the entire amount of data space
allocated. They just need it available for the worse-case. An example
is `vi` (`vim`) and emacs.
Certainly the current `kill-the-memory-hog` situation has not found
the correct solution. I think the kernel has to steal pages from the
memory-hog itself, writing them to the swap device/file. This keeps
a memory-hog out of everybody else's pants. But, the kernel doesn't
really 'know' very much about each process at the present time. I think
it will eventually have to keep track of the frequency of memory accesses
(per task) so it knows what pages to steal when memory is low. And it's
going to have to steal pages from the current task, not just some poor
sucker that's been sleeping.
What this would do is force a memory-hog to slow down because its pages
are being swapped, while better-behaved processes continue with little
or no impact.
Of course, if you don't have a swap-file, there is nothing you can
do. Some persons claim that a swap-file isn't necessary and should not
be used. They come from the "Flat Earth Society".
Cheers,
Dick Johnson
Penguin : Linux version 2.3.41 on an i686 machine (800.63 BogoMips).
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Wed Mar 15 2000 - 21:00:25 EST