On Thu, 16 Mar 2000, Paul Jakma wrote:
> On Thu, 16 Mar 2000, James Sutherland wrote:
>
> > On Wed, 15 Mar 2000, Paul Jakma wrote:
>
> > That doesn't happen. malloc() ALLOCATES the memory to the process. It is
> > *NOT* overcommitted. It may be backed by swapspace rather than physical
> > memory, but that block of memory *IS* available to the process.
>
> so malloc() isn't overcommited? malloc()'ed memory is guaranteed to be
> available - ie the memory is reserved and accounted for at malloc()
> time?
It CAN be, if you WANT it to. (/proc/sys/vm/overcommit). I don't have this
enabled, though, and I haven't changed this setting.
> (i always thought malloc()'s were noted by the kernel ie in the page
> table for the process, but that the kernel didn't actually allocate a
> real page until the process tried to access that page - ie malloc() is
> overcommited)
>
> If so, how do we get into trouble?
Something grabbed all the memory. Just running while(1){malloc(4096);}
will do this quite easily, unless you have process rlimits set to stop
that, in which case it is harder to demonstrate.
> > malloc() does this. fork() doesn't, because there is no memory to
> > allocate.
>
> i must be misunderstanding you, cause I'm pretty sure fork() is supposed
> to copy the complete VM of the parent. :) And on older Unix's like
> ultrix you really had to have lot's of swap if you wanted to allow big
> processes to do a fork().
It doesn't literally copy it all, though - it just marks it COW.
Otherwise, you would never be able to run a real WWW site with Apache, for
example - the memory demands would just be impossible to meet.
> > The whole point of fork() is that you are *NOT* simply
> > duplicating the in-VM image of the process!
>
> no, the whole point of overcommiting is so that we don't have to copy
> the VM of a process at fork() time, eg COW. But fork() does specify that
> the child process will have a complete copy of the parents VM.
Yes - what I meant is that we don't REALLY copy it all at fork() time -
that would be very inefficient. We only "pretend" to, via COW.
> (whether we copy it at fork() time like really ancient os's, COW with
> reserved backing store like ultrix or COW overcommit like linux is an
> implementation issue).
Yes - in practice, we really need the latter, though.
> > > but that only applies to processes that try malloc() at the point of
> > > OOM. You still have a bunch of processes with memory they have already
> > > malloc()'ed but havn't allocated yet.
> >
> > Nope.
>
> really really sure about this? cause it goes completely against my
> understanding of how linux works. but i'd be very glad to be corrected
> if i'm wrong. I've honestly thought that malloc() was overcommited up
> till now.
Unless you have specifically enabled this, malloc() shouldn't be
overcommitted - you call malloc() and get some memory, and it's yours.
It's only things like fork(), where you only have "fake" memory (it's
really shared with someone else until you write to it), that overcommit
takes effect otherwise.
James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:22 EST