Re: 1024-way SMP?! (was Re: linux-kernel-digest V1 #2914)

Paul Barton-Davis (pbd@Op.Net)
Tue, 01 Dec 1998 14:39:18 -0500


>> PBD said:
>>
>> I'd do it the same way that KSR did, except that I'd use off the shelf
>> processors. Cornell, I think, had a 1024 node KSR-1 and maybe even a
>> 1024 KSR-2. They never did get all the kinks out of their cache
>> coherency protocol, but I think that between them and Alewife, it
>> could probably be done.
>
>Hahahaha. Have you *any* idea how bad the KSR interconnect scheme is
>for SMP? ...
>
>Don't get me wrong: KSRs were lovely boxes for prototyping on, because
>they gave you the illusion of a big SMP system. However, their
>performance was simply not up to it, precisely because they tried to
>pretend that you didn't need to control communications explicitly at
>that sort of scale.

What KSR said about their machine is one thing. What it was is
something else. What it could have been is something else again.
The NUMA-with-pseudo-SMP model they presented was, as you note, an
excellent platform since you could prototype as if it was an SMP box
and then tune the NUMA-related bottlenecks.

The truth is, SMP doesn't scale to large-scale parallelism (I'm not
sure that *programming* scales to such things, but thats another
story), and so NUMA is the way to go. Offering the potential of an
SMP-view of things while making it still possible to do explicitly
NUMA-style work is great. The fact that they chose the wrong silicon,
had poorly manufactured components in key parts of the memory bus and
were always financially adrift is not relevant to the idea.

Expecting to get real performance from 1024 cpus in an SMP config is
not realistic. That why I said I'd do it the way KSR did - it will
work like an SMP box, and it will work better as a NUMA box. Too bad
it didn't work at all :)

--p

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/