Re: Avoiding OOM on overcommit...?

From: Linda Walsh (law@sgi.com)
Date: Thu Mar 30 2000 - 21:21:17 EST


"Peter T. Breuer" wrote:
> Perhaps you are confusing "not being able to start more processes" with
> "an existing process getting an unexpected signal due to an
> unobservable system condition not of its making or ever reported to
> it". The latter is OOM. It can never happen when no process can start
> unless there is swap space available to take its max stack (which
> guarantees that stack memory is never overcommitted), AND every access
> to malloc is wrappped as gnumalloc does it (which guarantess that heap
> memory can never be overcommitted).

---
	Mostly correct -- but you also would have to make sure all of your pages were
not COW(shared) which they are after a fork.  Why bother rewriting malloc?  Just change
'brk' to reflect this policy:

brk and sbrk are used to change dynamically the amount of space allocated for the calling process's data segment [see exec(2)]. The change is made by resetting the process's break value and allocating the appropriate amount of space. The break value is the address of the first location beyond the end of the data segment. The amount of allocated space increases as the break value increases.

Combine that with RESERVED memory for stack and all COW pages and you have your secure processes. (Note -- I am explicitly ignoring the case of a program running under debug or that is self-modifying). In all cases, I propose letting the kernel do RESERVED memory management. It should be centralized at the kernel level where the policy can be effectively administered -- since someone could have their own version of 'malloc' linked in with their code (not using the system shared malloc).

A possibly very safe development plan would be to first) just add the accounting of "Reserved" memory in brk/fork. 2nd) add virtual memory that would modify the remaining "virtual memory free" count upward if added. Both of those are 'safe' since they involve only 'numbers' - the 1st being accounting, the 2nd, lying to the kernel about free memory (in the same way it now overcommits). Third could go multiple ways. To get to a *safer* (not safe) mode would be to then allow the kernel to enforce reserved memory < free.

At this point, if 1T of virtual swap was added, we'd have the same behavior as today -- no changes. For those that want "safer" behavior one could run w/o virtual swap. Now your only un-safe allocation is for stack pages. To fix that in the short term -- a single process could call a subroutine that holds a 1 (or 8 or whatever) Meg array that it zero's out on startup. One time cost that Reserves physical memory for stack space.

For a long term fix, we need define an API that allows specifying stack space on a per/program and/or per/user and/or per/system basis. Additionally, specifying ulimits for Maximum process size might be desired.

-linda

-- Linda A Walsh | Trust Technology, Core Linux, SGI law@sgi.com | Voice: (650) 933-5338

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Mar 31 2000 - 21:00:28 EST