I would suggest that you take a look at the many, many papers on cache
affinity. All of these pares are saying one thing, and one thing only:
process migration is a very, very bad idea. It's the best way imaginable
to make a system of computers slow to a crawl.
Every SMP vendor would be happy (or not so happy) to tell you about their
efforts in process migration on SMP hardware, where you can imagine it is
substantially less painful to move the process. Less painful in terms of
OS code, but extremely painful in terms of hardware efficiency.
Good OS engineers learn to mentally view the idea of process migration as
a very, very exceptional event, one that you want to do only under duress.
This discussion is similar to the UFS vs LFS discussion. UFS tries hard to
put the data in the right plcae the first time, LFS tries to move it there
later. Needless to say, LFS sucks under load.
Process migration is to processes as the LFS cleaner is to file systems.
It's better to put your processes in the right place in the first place.
: What's the point of remote execution, if suddenly all
: those processes that were sleeping now use their time slice fully? I'm
: aware of (b), but I'll try to get the least involvement of the kernel.
If you really insist on this, you are simply reinventing the wheel. The
system that you describe has been done, 100% at user level, and has been
around for years. Condor does this. Jobs get sent to the "network" and
land on some computer that is idle. When the computer is used again (i.e.,
the mouse moves or some non-remote job app starts up) the process is
checkpointed and moved.
: > I had a version of make(1) that did all this stuff, I got a 45 minute
: > to 3.5 minute speedup on a cluster 8 SS2's plus a 4/470 file server.
: > Not bad, eh?
:
: Really not bad! If I can get the migration stuff working the way I want,
: the good old make would do very good just by spawning it's gccs on the
: same machine, without even suspecting that they'll end up being transfered
: somewhere else. :)
I think you are missing a fundamental optimization. If you "migrate"
only at exec(2) time, then you get the best of both worlds and the worst
of neither.