Seems to me that what PVM is doing is exactly the right thing to do.
Suppose I have N jobs and M CPUs where N >> M. I can distribute the load several different ways, the ends of the spectrum being:
. foreach job { startup job on CPU job % M }
OR
. job = busy = done = 0;
while (done < N) {
while (busy == M) {
waitforjob();
busy--;
done++;
}
startjob(job++);
busy++;
}
It's clear that if you do method #1, you'll need migration if jobs are not
all the same size. However, that's a silly point because why would you
do method #1? It creates a huge spike of new processes at the beginning
of the job, resulting in a thrashing system. Method #2 is (a) more
efficient, and (b) doesn't require any migration.
: > So are you just giving up on the efficiency issue? It's OK to throw
: > performance away because you are using DSM?
:
: Which is going to cost more - a clean DSM environment thats easy to get
: things running in or a hard to use but slightly more efficient MPI interface
: (and if we are going to argue efficiency you can stick PVM somewere damp
: and hit the flush).
So you are arguing that we should use DSM because it is as easier
programming model, right? And that it's OK to give up some performance
for the ease of use. OK, fine. Sounds like a reasonable argument.
Let's look at it for a minute, though.
In shared memory systems, all processors can access data with very
close to identical latencies. This is a cornerstone of the model,
would you not agree? It lets programmers have that ease of use that
you mentioned, they concern themselves only with the parallelization
and synchronization of their job (no small task, that).
In a DSM system, processors access memory one of two ways remote,
(aka slow) or local (aka fast). In order to have things work for
some reasonable definition of work, now programmers need to concern
themselves not only with the parallelization and synchronization, but
also with the real location of the memory. Right? So where's that
ease of use now?
: Cluster costs are diving, programmer time is not.
I couldn't agree more. Where we differ is that you think that DSM is
going to make programmers solve problems faster and I think that it
is going to make them solve problems slower. My experience with DSM
is that it adds an extra level of complexity to an already complex
programming model and that turns into more programmer time, not less.
The DSM model is great for textbook problems. If it were great for
real problems, then wouldn't we see people using one of the existing
implementations out there? This isn't a new idea.
Here's another data point: scientific programmers on SMP machines
frequently use MPI instead of shared memory. The programming model
is simple, fast, and it works. Doesn't that seem completely crazy?
Use message passing on a SMP? I wonder why they do that.
: I've read all the DIPC code (have you ?) and I think its close to meeting
: that stated aim.
Whether I've read it or not is completely beside the point - we are
talking about programming models, not programming model implementations.
But yes, Alan, I've read it. I've also read and used mether, the Rice
DSM, Erlichson's DSM, SCI, to mention a few others. I've worked with
most of the national labs on clusters, had extensive discussions with
100's of real cluster users working on real, not imagined, problems,
ranging from scientific fortran jobs, to jet engine simulations, to
market trading systems, and even parallel remote make (since that's
what I care about, of course).
I was having these debates before Linux existed. As far as I can tell,
nothing has changed. It's the same old "shared memory is easy" claim,
which strangely enough never seems to be made by the people that are
actually working in the clustering area.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu