Re: Overcommitable memory??

From: David Whysong (dwhysong@physics.ucsb.edu)
Date: Mon Mar 20 2000 - 07:53:25 EST


On Mon, 20 Mar 2000, Jesse Pollard wrote:
>On Sun, 19 Mar 2000, David Whysong wrote:

>>I don't like resource limits. Using resource limits is similar to not
>>having memory overcommit -- you waste a lot of system resources "just in
>>case", the kernel needs to do a lot more accounting, and it's just
>>horribly inefficient.
>
>Resource limits CAN prevent the OOM condition if
> 1. the sum of all concurrent users is <= total resources
> 2. users are not allowed to exceed their quota
        3. Kernel dynamic memory allocation is limited, which might be
           tricky when you consider loading modules, glx/DRI/agpgart
           stuff, soundcard buffers, etc.

Yes I realize the kernel reserves some pages for growth, but I am aware of
no guarantee that it won't need more than that reserve.

[...]
>>If init malfunctions, you're hosed no matter what.
>
>It only aborts (normal situation) if the system is allowed to go OOM.

This is a kernel bug.

>>>>The core problem remains. You, the user, have a finite amount of
>>>>memory available to you, however that limit is defined. Once you run
>>>>out, your processes will start dying.
>>
>>But that's not the problem! That's the way things have to be. You can't
>>use more resources than there are available.
>
>No you can't - what you are doing is "giving" access to resources you
>don't have. When the resources are then accessed - you die; and the system
>with it.

Obviously something has to die, but the system should not go down. This
behavior is a bug.

Kernel per-user memory accounting can protect against a DoS attack like a
fork/malloc bomb, and probably makes OOM situations more difficult to
encounter.

The cost is that every user can only access a fraction of the total
memory, which in practice leads to a situation analagous to no memory
overcommit -- you waste lots of VM. From a user's standpoint the OOM
problem happens much more frequently (similar to the no overcommit
situation, which actually causes OOM to happen sooner given a fixed amount
of VM) since each user can only access a fraction of the resources that
are available to the system.

>>>Yes - ME THE USER. I should not be able to cause the SYSTEM a problem.
>>>I should not be able to cause OTHER users a problem.
>>
>>Ahh, no! You can only do this by setting up horribly restrictive quotas
>>and effectively removing overcommit, which is terribly wasteful!
>
>not horribly - It does appear that way when you have never been forced to
>live within a budget.

For n users, you would have to limit each user to a fraction 1/n of the
total system VM in order to prevent an OOM situation (assuming you managed
to deal with all the nasty dynamic allocation issues somehow). That's
pretty restrictive for n > 1. IMNSHO.

>>Well, my OOM complaints stem from the fact that right now OOM situations
>>are functionally equivalent to crashing the machine.
>
>Can't be helped much, as long as resource control is missing.

Rik's patch helps an awful lot. Though I think it could use more
fine-tuning when accounting for the process "nice" level.

Dave

David Whysong dwhysong@physics.ucsb.edu
Astrophysics graduate student University of California, Santa Barbara
My public PGP keys are on my web page - http://www.physics.ucsb.edu/~dwhysong
DSS PGP Key 0x903F5BD6 : FE78 91FE 4508 106F 7C88 1706 B792 6995 903F 5BD6
D-H PGP key 0x5DAB0F91 : BC33 0F36 FCCD E72C 441F 663A 72ED 7FB7 5DAB 0F91

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:29 EST