Re: Overcommitable memory??

From: John Ripley (john@empeg.com)
Date: Wed Mar 22 2000 - 12:29:57 EST


Marco Colombo wrote:
> On Wed, 22 Mar 2000, John Ripley wrote:
> > main()
> > {
> > syscall_just_this_process_overcommit(0);
> >
> > char *p = (char *) malloc(1048576);
> > if(!p) {
> > printf("Hey we ran out of memory!\n");
> > return ENOMEM;
> > }
> > // this memory is guaranteed safe to acces
> > for(int i=0; i<1048576; i++) *p++ = i;
> > // some other stuff...
> > free(p);
> > return 0;
> > }
> >
> > To handle SIGBUS's is completely impractical. You would have to change
> > the interrupted address to a bailout address. Example with overcommit:
> >
> > void *bailout_addr = NULL;
> > void mysig() {
> > if(bailout_addr)
> > set_interrupted_address_somehow(bailout_addr);
> > }
> >
> > main()
> > {
> > char *p = (char *) malloc(1048576);
> > if(!p) goto bailout;
> >
> > bailout_addr = bailout;
> > signal(SIGBUS, mysig);
> > for(int i=0; i<1048576; i++) *p++ = i;
> > // etc...
> > free(p);
> > return 0;
> > bailout:
> > printf("Hey we ran out of memory!\n");
> > return ENOMEM;
> > }
>
> Nicely spotted.
>
> > Ignore the fact printf can run out of memory - in one of the apps I've
> > written I have a special version which can't.
>
> Do you pass arguments in registers, instead of stack?

Preallocated memory, but yes it makes an assumption that the stack is in
a fit state.

> Are you sure the variable 'bailout_addr' does not belong to a page
> that has been paged-out, at the time the signal-handler is invoked?
>
> If your application just exits, you don't need to catch the signal,
> you don't need the signal at all.
>
> If is tries to do something useful, it has to be sure it won't cause
> any other page-fault, or will make things just worse.
>
> Unless the kernel starts sending signals when swap utilization exceeds
> 75%.
> > Would you not agree that a SIGBUS is a completely silly way of finding
> > out you didn't get your memory? Now imagine this in a threaded app.
>
> You may start a GC() which does not need to know where you were when
> you received the signal. Problem is that the GC() has little space
> to run itself.
>

(I'm not sure I understand what you mean by a "GC")

This shows just how impractical it is to catch SIGBUS after malloc
(which is just an mmap with large enough chunks).

Here's a short list of problems:

- Signal handler may be paged out. Solution: mlock
- Jump location variable may be paged out. Solution: mlock
- Object instance cleanup needed, with much care taken about concurrency
issues.
- Hard to judge exactly where the fault occurred. Solution: COW all
pages after malloc, as it can only then happen in that operation. This
brings up:
- Inefficient.
- An absolute bastard to use. Especially if you're threaded
(/impossible)

It's quite ridiculous that anyone can advocate this kind of solution,
really.

-- 
John Ripley, empeg Ltd.
http://www.empeg.com

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:36 EST