Re: Endless overcommit memory thread.

From: Scott Henry (scotth@sgi.com)
Date: Fri Mar 24 2000 - 18:51:11 EST


>>>>> "M" == Mike A Harris <mharris@meteng.on.ca> writes:

M> Can someone please explain to me how and why an app would
M> allocate a large amount of memory and then not use it? Please
M> give specific examples since I can't fathom the reason. I think
M> an app that requests a lot of memory and then doesn't use it is
M> BAD - broken as designed. I can see legitimate allocation of
M> SMALL amounts of unused memory as speculation in time critical
M> code, or for optimization, etc.. but not large amounts.

2 examples off the top of my head:

1) fork/exec. The fork() effectively allocates again the VM of the
   original process. The exec() may be a much smaller one. The (rather
   extreme) example used in the IRIX newsgroups is the 100GB+
   simulation running for a week that needs to popen() (ie: fork/exec)
   a much smaller program (eg: ls) for some reason. You want to tell the
   customer that they need to buy double the memory and/or swap when a
   minor configuration change will make it work?

2) a lot of those large (FORTRAN) simulations allocate huge arrays,
   then use only a small amount of it, depending on the specifics of
   the problem that run. Yes, they could be recoded to use
   sparse-matrix techniques, but that takes time and resources that
   could be better used. And may actually run slower.

The kernel VM allocation system can't tell the difference between 1 & 2.

-- 
 Scott Henry <scotth@sgi.com> /  Help! My disclaimer is missing!
 IRIX MTS,                   /  GIGO *really* means: Garbage in, Gospel Out
 Silicon Graphics, Inc      /  http://reality.sgi.com/scotth/

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Mar 31 2000 - 21:00:14 EST