Re: Memory issues with Opteron 6220

[Jesper Krogh]

On 2012-02-09 09:33, Ingo Molnar wrote:
> * Anders Ossowicki<aowi@xxxxxxxxxxxxx>  wrote:
> > Hey,
> >
> > We're seeing unexpected slowdowns and other memory issues with a new system.
> > Enough to render it unusable. For example:
> >
> > Error: open3: fork failed: Cannot allocate memory
> >
> > at times where there's no real memory pressure:
> >                     total       used       free     shared    buffers     cached
> >        Mem:     132270720  131942388     328332          0     299768  103334420
> >        -/+ buffers/cache:   28308200  103962520
> >        Swap:      7811068      13760    7797308
> >
> > [...]
Anders' co-worker here..  below C-code (Summary: for -t processes that 
repeatedly
allocates and dallocates 2GB of memory) can excersize the bug
pretty frequently using -t 32 on this machine. On the other 128GB
machine it can run without issues.

It actually ended up toasting the machine:
jk@nysvin:~$ ./foo -t 32
-bash: fork: Cannot allocate memory
jk@nysvin:~$ w
-bash: fork: Cannot allocate memory
jk@nysvin:~$ top
-bash: fork: Cannot allocate memory
jk@nysvin:~$ ls
-bash: fork: Cannot allocate memory

I dont know what to conclude.

jk@nysvin:~$ ./foo -t 32
Upper bound: 1953 MB
malloc(1953) MB failed. iterations: 6
malloc(1953) MB failed. iterations: 2
malloc(1953) MB failed. iterations: 8

foo.c
#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <sys/wait.h>

void worker(void)
{
    long long i;
    char *p;
    int action;
    int mult = 500000;
    int size = mult * 4096;
    fprintf(stderr,"Upper bound: %lu MB\n",(long int) (size/1024/1024));
    for (i=0; ; i++) {
        action = i%2;
        switch(action) {
        case 0:
            p = malloc(size);
            if (!p){
                fprintf(stderr,"malloc(%lu) MB failed. iterations: 
%lli\n", (long int)size/1024/1024,i);
                exit(1);
            }
            break;
        case 1:
            free(p);
            break;
        }
    }
}

void usage(const char *cmd)
{
    fprintf(stderr,"Usage: %s [-t numthreads]\n", cmd);
    exit(1);
}

int main(int argc, char **argv)
{
    int c, i;
    int nproc = sysconf(_SC_NPROCESSORS_ONLN);


    while ((c = getopt(argc, argv, "t:")) != EOF) {
        switch (c) {
        case 't':
            nproc = strtol(optarg, 0, 0);
            break;
        default:
            usage(argv[0]);
        }
    }

    //printf("forking %d children\n", nproc);
    for (i=0; i < nproc; i++) {
        switch(fork()) {
        case -1:
            fprintf(stderr,"fork: %s\n", strerror(errno));
            exit(1);
        case 0: /* child */
            worker();
            exit(0);
        default: /* parent */
            /* nothing */
            break;
        }
    }

    for (i=0; i < nproc; i++) {
        int x, p;
        p = wait(&x);
    }

    return 0;
}

Can also be found here: http://shrek.krogh.cc/~jesper/foo.c

--
Jesper Krogh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/