Re: Overcomittable memory

From: John Ripley (john@empeg.com)
Date: Wed Mar 29 2000 - 11:37:09 EST

James Sutherland wrote:
> On 28 Mar 2000 00:26:37 GMT, you wrote:
> >On 21 Mar 2000 23:36:59 +0100, Marco Colombo <marco@esi.it> wrote:

> >reserving some memory only for kernel and/or root, etc, but it will just
> >lower the chances for OOM, not eliminate it.
> >
> >OOM will still happen, and we need some handling when it hits us. killing
> >random process is at least guaranteed to be fair as far as it goes, but
> >fairness is not necessarily the most sane thing to do in all cases...
>
> Not necessarily "killing" - just "signalling". Properly written, a
> program should be able to handle the signal and exit gracefully. If it
> doesn't handle a "something's wrong" signal, can we expect it to check
> every malloc() call for an equivalent return value?

Have you tried implementing a program that handles SIGBUS? It's a bloody
nightmare that will drive you insane (especially if you plan on using
threads).

I think I have previously sent an example pseudo-program demonstrating
this. But for other viewers in linux-kernel who may have missed it:

(I've modified this to say mmap instead of malloc to satisy some pedants
out there...) 

---

Example program where mmap returns ENOMEM deterministically at point of
allocation:

main()
{
        syscall_just_this_process_overcommit(0);

        char *p = (char *) mmap(NULL, 1048576, PROT_READ|PROT_WRITE,
				MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
        if(!p) {
                printf("Hey we ran out of memory!\n");
                return ENOMEM;
        }
        // this memory is guaranteed safe to acces
        for(int i=0; i<1048576; i++) *p++ = i;
        // some other stuff...
        free(p);
        return 0;
}

---

Example program where mmap returns memory regardless of ability to page.
SIGBUS returned non-deterministically and asynschronously:

void *bailout_addr = NULL;
void mysig() {
        if(bailout_addr)
                set_interrupted_address_somehow(bailout_addr);
}

main()
{
        char *p = (char *) mmap(NULL, 1048576, PROT_READ|PROT_WRITE,
				MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
        if(!p) goto bailout;

        bailout_addr = bailout;
        signal(SIGBUS, mysig);
        for(int i=0; i<1048576; i++) *p++ = i;
        // etc...
        free(p);
        return 0;
 bailout:
        printf("Hey we ran out of memory!\n");
        return ENOMEM;
}

Yes, the second example lacks sigstack etc, but you can assign fixed
stacks etc. In reality it's very hard to actually overrun your stack if
you write code knowing it's limited. Other problems include executable
pages being paged out and then faulted (==SIGBUS with handler on disk
not in memory). You can mlock() to get around that, but then you need to
be root, and mlock()'ing of executable pages from devices is broken
anyway.

---

Now, which of these two examples do you suppose is easier to write and
maintain? At any rate, a program which doesn't check the return value of
malloc/mmap etc is either a quick hack or it's plain BROKEN.

Nobody writes a SIGBUS handler because it's just not practical. If, on
the other hand, somebody can demonstrate a nice way of doing so, I would
be happy to use it :)

-- 
John Ripley, empeg Ltd.
http://www.empeg.com

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

This archive was generated by hypermail 2b29 : Fri Mar 31 2000 - 21:00:25 EST