No swap => pathological paging

Tim Hollebeek (tim@franck.Princeton.EDU)
Wed, 30 Apr 1997 10:44:59 -0400 (EDT)


2.0.30. Consider the program attached at the end. It allocates more memory
than I actually have VM (yes I know about overcommits and all the issues and
don't want to get started about that however ...) and then walks through it
writing.

Now, as long as swap space actually exists, the paging behavior of this is
quite good, since it uses memory sequentially. However, as soon as swap
space hits zero, the new pages being created *cannot* be transfered out of
main memory. Linux valiantly tries to continue, swapping out X, libc, and
virtually everything else. The practical outcome of this is that when swap
space hits zero, my machine goes from a quite usable condition (>1 mouse
interrupt per second, for example) to completely useless (<1 mouse interrupt
per 10 minutes) in under a second!

One assumes evidentally memory allocation will fail and things will
start dying and things will return to normal. This does happen, but
only after about 4 hours.

Doing the experiment again with the sleep() uncommented allows one to
watch the process in slow motion confirming what is going on; my
system has 30864k _real_ memory and I've managed to watch the process
until 28000k is occupied by the program. (yes, that's two xterms, top,
and X running in 2300k! Performance is less than ideal :=))

It should be possible to do much better than this especially with such a
swap-friendly program; I've been poking through vmscan.c to look for good
places to fix this, but haven't come up with anything so I was wondering
what the opinions of the linux gurus are.

Program:

#include <stdlib.h>
#include <stdio.h>

double **new_matrix(int n) {
double **ret = (double **)malloc(n * sizeof(double *));
int i;

for (i = 0; i < n; i++)
ret[i] = (double *)malloc(n*sizeof(double));

return ret;
}

int main() {
double **m = new_matrix(3300);
int i, j;

for (i = 0; i < 3300; i++) {
for (j = 0; j < 3300; j++) {
m[i][j] = 1;
}

printf("%d\n", i);
fflush(0);
/* if (i > 1650) sleep(5); */
}
}

---------------------------------------------------------------------------
Tim Hollebeek | Disclaimer :=> Everything above is a true statement,
Electron Psychologist | for sufficiently false values of true.
Princeton University | email: tim@wfn-shop.princeton.edu
----------------------| http://wfn-shop.princeton.edu/~tim (NEW! IMPROVED!)