Horst von Brand writes:
> Olaf Weber <olaf@infovore.xs4all.nl> said:
> [...]
>> Read-only data is not a problem, so apart from fork/exec, how many
>> cases are there where you have processes sharing large numbers of
>> writable pages? Note that for overcommitment to actually "work" in
>> those cases, those pages should hardly ever be written to: if they are
>> all touched in the long run, then you do really need the extra memory,
>> and reserving it now will prevent nasty surprises later. And if the
>> pages are de-facto read-only, would it not be better if the
>> application marked them as such before forking?
> Many architectures don't know of read-only data, so you must
> consider them writeable anyway.
If you can implement copy on write, you can implement abort on write.
And if you cannot implement copy on write, how do you implement
overcommitment?
> Shared libraries often live in writeable segments, as addresses need
> to be fixed up.
Which means that even an overcommitting system can only share pages
between processes where the addresses were fixed up in the same way.
If run into this kind of thing, you can still make the shared library
read-only after the address fixup.
> And it is the case that writeable stuff is shared (on fork(2), each
> single time). Might be short-lived, but without overcommit it HAS to
> be available _now_. With overcommit you can hope that not every
> other user does cash in immediately on reserved memory, or that some
> other process exits and frees some before you need it.
For fork/exec, you might want to have a "special" fork which is safer
than vfork but doesn't imply the need to reserve a copy of all the
writable data. Or, as someone proposed, you fudge fork to allow a
limited form of overcommit until the next system call.
It was pointed out that stack growth is another problem, and it is
indeed one where a failure is even harder to recover from than from a
malloc failure.
> Marking "de-facto read only by the process" is error-prono, and will
> be done wrong enough of the time it will be taken back. OOM is rare,
> program chrashing because "I thought read-only but overlooked..."
> will be frequent.
It would be the program itself that marks pages read-only, so if these
pages are subsequently written to it either means they shouldn't have
been read-only (and the extra memory was needed after all), or there
was a hidden bug in program.
> Next time go ask your bank if they have the cash handy for the case that
> everybody decides to close their accounts the same day.
Actually banking is not a bad analogy: just remember that they are
subject to strict rules w.r.t. to the extent to which their cash
reserves may be overcommitted, and are also obliged to have the assets
to cover _all_ commitments. Banks also differentiate between good and
bad risks.
In terms of risk it makes a real difference if you overcommitted 100MB
of de-facto read-only COW memory, or 10MB of de-facto read-write
memory: the latter commitment is the one that kills.
In general, my point is just that some advocates of overcommitment
exaggerate the amount of extra VM that a non-overcommitting system
would require. An OOM situation is bad on both systems, though I
believe programs have a better chance of coping with it on a system
that doesn't overcommit.
If you have the infrastructure necessary to prevent overcommitment,
you are also in a position to allow it to a limited extent, or in
certain special cases. Without that infrastructure, no-one has a
choice.
-- Olaf WeberDo not meddle in the affairs of sysadmins, for they are quick to anger and have no need for subtlety.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:26 EST