Re: New NUMA scheduler and hotplug CPU

[Nick Piggin]

Martin J. Bligh wrote:

> > > 	Looking at your new scheduler in -mm, it uses cpu_online_map
> > > alot in arch_init_sched_domains.  This means with hotplug CPU that it
> > > would need to be modified: certainly possible to do, but messy.
> > >
> > > 	The other option is to use cpu_possible_map to create the full
> > > topology up front, and then it need never change.  AFAICT, no other
> > > changes are neccessary: you already check against moving tasks to
> > > offline cpus.
> > >
> > > Anyway, I was just porting the hotplug CPU patches over to -mm, and
> > > came across this, so I thought I'd ask.
> > Hi Rusty,
> > Yes I'd like to use the cpu_possible_map to create the full
> > topology straight up. Martin?
>
> Well isn't it a bad idea to have cpus in the data that are offline?
> It'll throw off all your balancing calculations, won't it? You seemed
> to be careful to do things like divide the total load on the node by
> the number of CPUs on the node, and that'll get totally borked if you
> have fake CPUs in there.

I think it mostly does a good job at making sure to only take
online cpus into account. If there are places where it doesn't
then it shouldn't be too hard to fix.


> To me, it'd make more sense to add the CPUs to the scheduler structures
> as they get brought online. I can also imagine machines where you have
> a massive (infinite?) variety of possible CPUs that could appear - 
> like an NUMA box where you could just plug arbitrary numbers of new
> nodes in as you wanted.

I guess so, but you'd still need NR_CPUS to be >= that arbitrary
number.

> Moreover, as the CPUs aren't fixed numbers in advance, how are you going 
> to know which node to put them in, etc? Setting up every possible thing 
> in advance seems like an infeasible way to do hotplug to me. 

Well this would be the problem. I guess its quite possible that
one doesn't know the topology of newly added CPUs before hand.

Well OK, this would require a per architecture function to handle
CPU hotplug. It could possibly just default to arch_init_sched_domains,
and just completely reinitialise everything which would be the simplest.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/