Re: New NUMA scheduler and hotplug CPU

[Nick Piggin]

Martin J. Bligh wrote:

> > > Well isn't it a bad idea to have cpus in the data that are offline?
> > > It'll throw off all your balancing calculations, won't it? You seemed
> > > to be careful to do things like divide the total load on the node by
> > > the number of CPUs on the node, and that'll get totally borked if you
> > > have fake CPUs in there.
> > I think it mostly does a good job at making sure to only take
> > online cpus into account. If there are places where it doesn't
> > then it shouldn't be too hard to fix.
>
> It'd make the code a damned sight simpler and cleaner if you dropped
> all that stuff, and updated the structures when you hotplugged a CPU,
> which is really the only sensible way to do it anyway ...
>
> For instance, if I remove cpu X, then bring back a new CPU on another node
> (or in another HT sibling pair) as CPU X, then you'll need to update all
> that stuff anyway. CPUs aren't fixed position in that map - the ordering
> handed out is arbitrary.
>
>
> > > To me, it'd make more sense to add the CPUs to the scheduler structures
> > > as they get brought online. I can also imagine machines where you have
> > > a massive (infinite?) variety of possible CPUs that could appear - 
> > > like an NUMA box where you could just plug arbitrary numbers of new
> > > nodes in as you wanted.
> > I guess so, but you'd still need NR_CPUS to be >= that arbitrary
> > number.
>
> Yup ... but you don't have to enumerate all possible positions that way.
> See Linus' arguement re dynamic device numbers and ISCSI disks, etc.
> Same thing applies.
>
>
> > Well this would be the problem. I guess its quite possible that
> > one doesn't know the topology of newly added CPUs before hand.
> >
> > Well OK, this would require a per architecture function to handle
> > CPU hotplug. It could possibly just default to arch_init_sched_domains,
> > and just completely reinitialise everything which would be the simplest.
>
> Yeah, it's not trivially simple. But then neither is the rest of CPU 
> hotplug, to do it right ;-) Requiring CPU hotplug callback hooks does 
> seem to be the right way to interface with the sched code though ...

OK you've convinced me.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/