Re: Whining about NUMA. :) [Was whining about 2.5...]

From: Marcelo Tosatti (
Date: Mon Oct 08 2001 - 12:48:49 EST

On Mon, 8 Oct 2001, Martin J. Bligh wrote:

> >> The worst possible case I can conceive (in the future architectures
> >> that I know of) is 4 different levels. I don't think the number of access
> >> speed levels is ever related to the number of processors ?
> >> (users of other NUMA architectures feel free to slap me at this point).
> >
> > So you're saying that at most any given node is 4 hops away from any
> > other for your arch?
> For the current architecture (well, for NUMA-Q) it's 0 or 1. For future
> architectures, there will be more (forgive me for deliberately not being
> specific ... I'd have to ask for more blessing first). Up to about 4. Ish.
> Depending on how much extra latency each hop introduces, it may well
> not be worth adding the complexity of differentiating beyond local vs
> remote? At least at first ...
> Do you know how many hops SGI can get, and how much extra latency
> you introduce? I know we're something like 10:1 ratio at the moment
> between local and remote.
> I guess my main point was that the number of levels was more like constant
> than linear. Maybe for large interconnected switched systems with small
> switches, it's n log n, but in practice I think log n is small enough to be
> considered constant (the number of levels of switches).
> >> So I *think* the worst possible case is still linear (to number of nodes)
> >> in terms of how many classzone type things we'd need? And the number
> >> of classzone type things any given access would have to search through
> >> for an access is constant? The number of zones searched would be
> >> (worst case) linear to number of nodes?
> >
> > That's how we have our stuff coded at the moment, but with classzones you
> > might be able to get that down even further. For instance, you could have
> > classzones that correspond to the number of hops a set of nodes is from a
> > given node. Having such classzones might make finding nearby memory easier.
> That's what I was planning on ... we'd need m x n classzones, where m
> was the number of levels, and n the number of nodes. Each search would
> obviously be through m classzones. I'll go poke at the current code some more.

You say "numbers of levels" as in each level being a given number of nodes
on that "level" distance ?

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

This archive was generated by hypermail 2b29 : Mon Oct 15 2001 - 21:00:19 EST