RE: [patch V2 38/38] x86/smpboot/64: Implement arch_cpuhp_init_parallel_bringup() and enable it

[Michael Kelley (LINUX)]

From: Thomas Gleixner <tglx@xxxxxxxxxxxxx> Sent: Saturday, May 6, 2023 9:23 AM
> 
> On Sat, May 06 2023 at 00:53, Michael Kelley wrote:
> > From: Thomas Gleixner <tglx@xxxxxxxxxxxxx> Sent: Thursday, May 4, 2023 12:03 PM
> > [snip]
> >
> >> @@ -934,10 +961,10 @@ static void announce_cpu(int cpu, int ap
> >>  	if (!node_width)
> >>  		node_width = num_digits(num_possible_nodes()) + 1; /* + '#' */
> >>
> >> -	if (cpu == 1)
> >> -		printk(KERN_INFO "x86: Booting SMP configuration:\n");
> >> -
> >>  	if (system_state < SYSTEM_RUNNING) {
> >> +		if (num_online_cpus() == 1)
> >
> > Unfortunately, this new check doesn't work.  Here's the output I get:
> >
> > [    0.721384] smp: Bringing up secondary CPUs ...
> > [    0.725359] smpboot: x86: Booting SMP configuration:
> > [    0.729249] .... node  #0, CPUs:        #2
> > [    0.729654] smpboot: x86: Booting SMP configuration:
> > [    0.737247]       #4
> >
> > Evidently num_online_cpus() isn't updated until after all the primary
> > siblings get started.
> 
> Duh. Where is that brown paperbag?
> 
> > When booting with cpuhp.parallel=0, the output is good.
> 
> Exactly that was on the command line when I quickly booted that change :(
> 
> The below should fix it for real.
> 
> Thanks,
> 
>         tglx
> ---
> --- a/arch/x86/kernel/smpboot.c
> +++ b/arch/x86/kernel/smpboot.c
> @@ -951,9 +951,9 @@ static int wakeup_secondary_cpu_via_init
>  /* reduce the number of lines printed when booting a large cpu count system */
>  static void announce_cpu(int cpu, int apicid)
>  {
> +	static int width, node_width, first = 1;
>  	static int current_node = NUMA_NO_NODE;
>  	int node = early_cpu_to_node(cpu);
> -	static int width, node_width;
> 
>  	if (!width)
>  		width = num_digits(num_possible_cpus()) + 1; /* + '#' sign */
> @@ -962,7 +962,7 @@ static void announce_cpu(int cpu, int ap
>  		node_width = num_digits(num_possible_nodes()) + 1; /* + '#' */
> 
>  	if (system_state < SYSTEM_RUNNING) {
> -		if (num_online_cpus() == 1)
> +		if (first)
>  			pr_info("x86: Booting SMP configuration:\n");
> 
>  		if (node != current_node) {
> @@ -975,11 +975,11 @@ static void announce_cpu(int cpu, int ap
>  		}
> 
>  		/* Add padding for the BSP */
> -		if (num_online_cpus() == 1)
> +		if (first)
>  			pr_cont("%*s", width + 1, " ");
> +		first = 0;
> 
>  		pr_cont("%*s#%d", width - num_digits(cpu), " ", cpu);
> -
>  	} else
>  		pr_info("Booting Node %d Processor %d APIC 0x%x\n",
>  			node, cpu, apicid);

This works.  dmesg output is clean for these guest VM combinations
on Hyper-V that I tested:

* Normal VM:  16 vCPUs in 1 NUMA node and 32 vCPUs in 2 NUMA nodes
* Same configs for a SEV-SNP Confidential VM with paravisor

Tested with and without cpuhp.parallel=0

For the entire series:
Tested-by: Michael Kelley <mikelley@xxxxxxxxxxxxx>