Re: Avoiding OOM on overcommit...?

From: Richard Gooch (rgooch@ras.ucalgary.ca)
Date: Wed Mar 29 2000 - 19:08:17 EST


Linda Walsh writes:
> Richard Gooch wrote:
> >
> > Linda Walsh writes:
> > > Richard Gooch wrote:
> > > > Each of these options is flawed:
> > > >
> > > > 1&2) you have to do this for all processes
> >
> > [bugger, you've removed the context]
> ---
> Sorry about that...:-)
> >
> > > You have to assign process ID's, memory mappings, etc to every
> > > process. So what's your point? Having a default guaranteed stack
> >
> > No, *I* don't have to allocate PIDs. The kernel does that for me.
>
> The kernel would reserve physical memory for stack as it does
> now. Just that right now, that is set to the actual page used, so,
> 1 page? I'm saying that could be a tuneable upward. So you don't
> *have* to reserve more space. You just have the choice to.

Yeah, right now the kernel "automatically" reserves 1 page for the
user stack anyway (simply because the process main() function will
require it).

Reserving some minimum number of pages has the problem of determining,
ahead of time, how many should be reserved. Are you suggesting this be
done on a case-by-case basis?

> > To avoid overcommit, I have to limit the stacksize for all
> > processes.
>
> Again, it's not a limit. It's a *minimum* guaranteed to
> physically be there (i.e. -- not overcommitted).

Still seems a shaky foundation. To make this work, you would at least
need a signal that is sent when you reach some high watermark of used
stack pages, but still have enough reserved so that the application
can still do something about it. SIGSTACKDANGER or some such.

> > I either set a global value which is big enough for my
> > monster simulation, or I go and hand-tune a pile of exceptions. And
> > from time to time get a segfault because I set the limit too low.
>
> You would only segfault if you physically ran out of memory,
> like now. If there is enough memory, the kernel does an 'on-demand'
> allocation. In non-OOM cases, it succeeds, and everything is
> normal...

OK, that's a lot better than forcing limits all over the place.

> > Right now I can safely set the global limit to 128 MiB, and not expect
> > any process to exceed that. If we abolish overcommitting, I've got to
> > reserve an awful lot of swap space now. 128 MiB * nproc mounts up
> > *very* quickly.
>
> No no no...you are misunderstanding. You can still have the
> exact same behavior today by allowing whatever the default stack
> size is today to be physically allocated (1 page?), and setting your
> virtual swap to infinite. You wouldn't have to change a thing.
> That's the beauty of this -- it's about *flexibility*. You get your
> behavior, I get mine, and anyone else gets any in-between behaviors
> they desire.

It would be even better if this mechanism wasn't restricted to stack
pages. Rather, just make it global, for all the process pages. Simply
reserve a specified number of pages, and send a signal when a high
watermark is reached.

> > That's why I said "handle". If I meant "killed by" I would have said
> > "killed by". For a stack exhaustion signal to be useful to my
> > application as a memory management tool (like you argue NULL return
> > from malloc(3) can be), I have to *handle* that signal and *do
> > something constructive*.
>
> This was a question I asked earlier and I didn't get a clear
> answer: is current context of a process saved on kernel or user
> stack before a user signal handler is called. Regardless, you are
> right if it is user stack and the behavior is no worse than today
> (unless you regard sleeping

I haven't checked, but it might be saved on the user stack, prior to
the kernel dispatching the signal handler. The signal handler will
definately consume user stack space.

                                Regards,

                                        Richard....
Permanent: rgooch@atnf.csiro.au
Current: rgooch@ras.ucalgary.ca

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Mar 31 2000 - 21:00:26 EST