Re: Whining about NUMA. :) [Was whining about 2.5...]

From: Martin J. Bligh (
Date: Mon Oct 08 2001 - 13:55:45 EST

>> The worst possible case I can conceive (in the future architectures
>> that I know of) is 4 different levels. I don't think the number of access
>> speed levels is ever related to the number of processors ?
>> (users of other NUMA architectures feel free to slap me at this point).
> So you're saying that at most any given node is 4 hops away from any
> other for your arch?

For the current architecture (well, for NUMA-Q) it's 0 or 1. For future
architectures, there will be more (forgive me for deliberately not being
specific ... I'd have to ask for more blessing first). Up to about 4. Ish.

Depending on how much extra latency each hop introduces, it may well
not be worth adding the complexity of differentiating beyond local vs
remote? At least at first ...

Do you know how many hops SGI can get, and how much extra latency
you introduce? I know we're something like 10:1 ratio at the moment
between local and remote.

I guess my main point was that the number of levels was more like constant
than linear. Maybe for large interconnected switched systems with small
switches, it's n log n, but in practice I think log n is small enough to be
considered constant (the number of levels of switches).

>> So I *think* the worst possible case is still linear (to number of nodes)
>> in terms of how many classzone type things we'd need? And the number
>> of classzone type things any given access would have to search through
>> for an access is constant? The number of zones searched would be
>> (worst case) linear to number of nodes?
> That's how we have our stuff coded at the moment, but with classzones you
> might be able to get that down even further. For instance, you could have
> classzones that correspond to the number of hops a set of nodes is from a
> given node. Having such classzones might make finding nearby memory easier.

That's what I was planning on ... we'd need m x n classzones, where m
was the number of levels, and n the number of nodes. Each search would
obviously be through m classzones. I'll go poke at the current code some more.


To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

This archive was generated by hypermail 2b29 : Mon Oct 15 2001 - 21:00:19 EST