Re: [PATCH v2 0/9] Introduce SMT level and add PowerPC support

From: Sachin Sant
Date: Wed Jun 28 2023 - 11:35:16 EST




> On 28-Jun-2023, at 3:35 PM, Laurent Dufour <ldufour@xxxxxxxxxxxxx> wrote:
>
> I'm taking over the series Michael sent previously [1] which is smartly
> reviewing the initial series I sent [2]. This series is addressing the
> comments sent by Thomas and me on the Michael's one.
>
> Here is a short introduction to the issue this series is addressing:
>
> When a new CPU is added, the kernel is activating all its threads. This
> leads to weird, but functional, result when adding CPU on a SMT 4 system
> for instance.
>
> Here the newly added CPU 1 has 8 threads while the other one has 4 threads
> active (system has been booted with the 'smt-enabled=4' kernel option):
>
> ltcden3-lp12:~ # ppc64_cpu --info
> Core 0: 0* 1* 2* 3* 4 5 6 7
> Core 1: 8* 9* 10* 11* 12* 13* 14* 15*
>
> This mixed SMT level may confused end users and/or some applications.
>

Thanks for the patches Laurent.

Is the SMT level retained even when dynamically changing SMT values?
I am observing difference in behaviour with and without smt-enabled
kernel command line option.

When smt-enabled= option is specified SMT level is retained across
cpu core remove and add.

Without this option but changing SMT level during runtime using
ppc64_cpu —smt=<level>, the SMT level is not retained after
cpu core add.

[root@ltcden8-lp8 ~]# ppc64_cpu —smt=4
[root@ltcden8-lp8 ~]# ppc64_cpu --info
Core 0: 0* 1* 2* 3* 4 5 6 7
Core 1: 8* 9* 10* 11* 12 13 14 15
Core 2: 16* 17* 18* 19* 20 21 22 23
Core 3: 24* 25* 26* 27* 28 29 30 31

Remove a core, SMT level is retained.

[root@ltcden8-lp8 ~]# ppc64_cpu --info
Core 0: 0* 1* 2* 3* 4 5 6 7
Core 1: 8* 9* 10* 11* 12 13 14 15
Core 2: 16* 17* 18* 19* 20 21 22 23
[root@ltcden8-lp8 ~]#

Add 3 cores, SMT level is not retained.

[ 496.600648] Fallback order for Node 1: 1 0
[ 496.600655] Built 1 zonelists, mobility grouping on. Total pages: 1228159
[ 496.600676] Policy zone: Normal
[ 496.661173] WARNING: workqueue cpumask: online intersect > possible intersect
[ 499.530646] Fallback order for Node 3: 3 0
[ 499.530655] Built 2 zonelists, mobility grouping on. Total pages: 1228159
[ 499.530675] Policy zone: Normal

ppc64_cpu --info
Core 0: 0* 1* 2* 3* 4 5 6 7
Core 1: 8* 9* 10* 11* 12 13 14 15
Core 2: 16* 17* 18* 19* 20 21 22 23
Core 3: 24* 25* 26* 27* 28* 29* 30* 31*
Core 4: 32* 33* 34* 35* 36* 37* 38* 39*
Core 5: 40* 41* 42* 43* 44* 45* 46* 47*
[root@ltcden8-lp8 ~]#

- Sachin