Re: Recursive malloc crashing Linux. (Well almost)

John Ripley (john@empeg.com)
Thu, 16 Dec 1999 13:31:59 +0000


--Boundary-=_nWlrBbmQBhCDarzOwKkYHIDdqSCD
Content-Type: text/plain
Content-Transfer-Encoding: 8bit

On Thu, 16 Dec 1999, you wrote:
> [John Ripley <john@empeg.com>]
> > I've encountered this problem too, but with the additional
> > restriction of having an environment with 8MB of RAM and no swap.
> > I'm not sure I agree with the ability to allocate more memory than
> > you have available.
>
> echo 0 > /proc/sys/vm/overcommit_memory
>
> grep -A10 overcommit /usr/src/linux/Documentation/proc.txt
>
> Peter

Let's change that slightly:

# cat /proc/sys/vm/overcommit_memory
0
# grep -A10 overcommit Documentation/proc.txt
overcommit_memory
This file contains one value. The following algorithm is used to
decide if there's enough memory: if the value of overcommit_memory
is positive, then there's always enough memory. This is a useful
feature, since programs often malloc() huge amounts of memory 'just
in case', while they only use a small part of it. Leaving this
value at 0 will lead to the failure of such a huge malloc(), when
in fact the system has enough memory for the program to run.

On the other hand, enabling this feature can cause you to run out
of memory and thrash the system to death, so large and/or important
servers will want to set this value to 0.

As you can see, overcommit defaults to off. Under such conditions, malloc
is supposed to fail gracefully if enough memory cannot be allocated.
However, you can quite happily allocate 1GB, so long as you allocate it in
chunks of memory smaller than your physical limit. I've included a short
demonstration program for this. On an empeg (8MB ram, no swap) this results
in:

# ./killmem
Allocating 256 amounts of 4MB == 1024MB
Successful
Filling block 0
Filling block 1
killmem: memory violation at pc=0x0200047c, lr=0x4005f7d8 (bad address=0x4065d000, code 1)
Bus error

Which is kind of not very graceful. On my PC, 128MB ram, ~133MB swap, it
gets as far as block 34 (136MB) before causes a complete lockup (not even
magic sysrq works) - which is also not very graceful. The empeg runs
2.2.12, the PC runs 2.3.30. A solution I've thought of would be to have
vm_enough_memory() (in mm/mmap.c) also take into account anonymous mmap
pages that are yet to be paged, which requires another global count such as
nr_pending_anon_pages.

Has anyone got any other solutions to this?

--
John Ripley, empeg Ltd.
http://www.empeg.com

--Boundary-=_nWlrBbmQBhCDarzOwKkYHIDdqSCD Content-Type: text/x-c; name="killmem.c" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="killmem.c"

I2luY2x1ZGUgPHN0ZGlvLmg+CiNpbmNsdWRlIDx1bmlzdGQuaD4KI2luY2x1ZGUgPHN5cy9tbWFu Lmg+CgojZGVmaW5lIE5CVUZTCQkyNTYKI2RlZmluZSBNTUFQX1NJWkUJKDQqMTA0ODU3NikKCm1h aW4oKQp7CiAgICBpbnQgaSwgajsKICAgIGNoYXIgKmJ1ZltOQlVGU10sICpwOwoKICAgIHByaW50 ZigiQWxsb2NhdGluZyAlZCBhbW91bnRzIG9mICVkTUIgPT0gJWRNQlxuIiwKCSAgIE5CVUZTLCBN TUFQX1NJWkUgPj4gMjAsIChNTUFQX1NJWkUgPj4gMjApICogTkJVRlMpOwogICAgCiAgICBmb3Io aT0wOyBpPE5CVUZTOyBpKyspIHsKCWJ1ZltpXSA9IChjaGFyICopCgkgICAgbW1hcCgwLCA0MDk2 KjEwMjQsIFBST1RfUkVBRHxQUk9UX1dSSVRFLAoJCSBNQVBfQU5PTllNT1VTfE1BUF9QUklWQVRF LCAtMSwgMCk7CglpZihidWZbaV0gPT0gTUFQX0ZBSUxFRCkgewoJICAgIHByaW50ZigiRmFpbGVk IGZvciBibG9jayAlZFxuIiwgaSk7CgkgICAgcmV0dXJuIDE7Cgl9CiAgICB9CgogICAgcHJpbnRm KCJTdWNjZXNzZnVsXG4iKTsKICAgIAogICAgZm9yKGk9MDsgaTxOQlVGUzsgaSsrKSB7Cglwcmlu dGYoIkZpbGxpbmcgYmxvY2sgJWRcbiIsIGkpOwoJCglwID0gYnVmW2ldOwoJZm9yKGo9TU1BUF9T SVpFOyBqOyBqLS0pICpwKysgPSAwOwogICAgfQoKICAgIHByaW50ZigiRG9uZVxuIik7Cn0K

--Boundary-=_nWlrBbmQBhCDarzOwKkYHIDdqSCD--

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/