Re: [PATCH v5 6/8] x86/resctrl: Sub NUMA Cluster detection and enable

[Reinette Chatre]

Hi Tony,

On 9/28/2023 11:12 AM, Tony Luck wrote:
> On Mon, Sep 25, 2023 at 04:29:44PM -0700, Reinette Chatre wrote:
>> On 8/29/2023 4:44 PM, Tony Luck wrote:

>>> +	if (cpumask_first(topology_core_cpumask(cpu)) != cpu)
>>> +		return;
>>> +
>>
>> This is an area I am not familiar with. The above code seems
>> to assume that CPUs are onlined in a particular numerical
>> order. For example, if I understand correctly, if CPUs
>> are onlined from higher number to lower number then
>> the above code may end up running on every CPU online.
> 
> This sent me on a voyage of exploration into early Linux
> bringup. There's a CONFIG_HOTPLUG_PARALLEL option to bring
> CPUs up in parallel. I have it set on my kernel, but I still
> see the "announce_cpu()" console messages show up in monotonic
> increasing order:
> 
> [    3.148423] smpboot: x86: Booting SMP configuration:
> [    3.148940] .... node  #0, CPUs:          #1   #2   #3   #4   #5   #6 #7 and so on
> 
> But, without solving this mystery, I realized this doesn't matter.
> Whatever order the CPUs come online was all completed long before
> resctrl is initialized:
> 
> 	late_initcall(resctrl_late_init);
> 
> So the order that resctrl sees CPUs is dependent on the callbacks
> from the registration with cpuhp_setup_state(). That happens with:
> 
>         /*
>          * Try to call the startup callback for each present cpu
>          * depending on the hotplug state of the cpu.
>          */
>         for_each_present_cpu(cpu) {
> 
> which is going to call in increasing numerical order as the bitmap
> of present CPUs is traversed.
> 
> If someone changed this, the only ill effect on the code I'm
> adding would be to set the MSR multiple times (which is
> inefficient, but won't break anything).
> 
> 

Thank you very much for investigating this.


Reinette