Re: [linus:master] [sched/numa] fc137c0dda: autonuma-benchmark.numa01.seconds 118.9% regression

[Raghavendra K T]

On 5/10/2023 1:25 PM, kernel test robot wrote:
> Hello,
>
> kernel test robot noticed a 118.9% regression of autonuma-benchmark.numa01.seconds on:
>
>
> commit: fc137c0ddab29b591db6a091dc6d7ce20ccb73f2 ("sched/numa: enhance vma scanning logic")
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>
> testcase: autonuma-benchmark
> test machine: 88 threads 2 sockets Intel(R) Xeon(R) Gold 6238M CPU @ 2.10GHz (Cascade Lake) with 128G memory
> parameters:
>
> 	iterations: 4x
> 	test: numa02_SMT
> 	cpufreq_governor: performance
>
>
> In addition to that, the commit also has significant impact on the following tests:
>
> +------------------+------------------------------------------------------------------------------------------------+
> | testcase: change | autonuma-benchmark: autonuma-benchmark.numa01.seconds 39.3% regression                         |
> | test machine     | 224 threads 2 sockets (Sapphire Rapids) with 256G memory                                       |
> | test parameters  | cpufreq_governor=performance                                                                   |
> |                  | iterations=4x                                                                                  |
> |                  | test=numa02_SMT                                                                                |
> +------------------+------------------------------------------------------------------------------------------------+
> | testcase: change | autonuma-benchmark: autonuma-benchmark.numa01.seconds 48.9% regression                         |
> | test machine     | 88 threads 2 sockets Intel(R) Xeon(R) Gold 6238M CPU @ 2.10GHz (Cascade Lake) with 128G memory |
> | test parameters  | cpufreq_governor=performance                                                                   |
> |                  | debug-setup=no-monitor                                                                         |
> |                  | iterations=4x                                                                                  |
> |                  | test=numa02_SMT                                                                                |
> +------------------+------------------------------------------------------------------------------------------------+
[...]

Hello,

Thanks for the detailed analysis. I have posted a RFC patch to address
this issue [1]. (that patch needs windows = 0 initialized FYI if needs
to be applied).  will be posting RFC V2 soon. Will add your reported-by
to that patchset. But one thing to note is [1] will be bringing back
*some* of the system overhead of vma scanning.

Here are some observations/Clarifications on numa01 test:

- numa01 benchmark improvements I got for numascan improvement patchset
[2] were based on mmtests' numa01, lets call  mmtest_numa01.
(some how this is not run in LKP ?)

- lkp_numa01 = mmtests' numa01_THREAD_ALLOC case mentioned in the
patch[1]

With numa scan enhancement patches there is a huge improvement regarding
system time overhead of vma scanning since we filter out scanning by
tasks which have not accessed VMA. This has benefited mmtest_numa01

However in case of lkp_numa01 we are observing that less PTE updates
happening because of filtering. (we can say a corner case of disjoint
set vma). This has caused regression you have reported.

backup:
----------
lkp_numa01:
3GB allocated memory that is distributed evenly to threads (24MB chunk).
24MB is then bzeroed by each thread 1000 times
mmtest_numa01:
entire 3GB bzeroed by all threads 50 times

[1]. 
https://lore.kernel.org/lkml/cover.1683033105.git.raghavendra.kt@xxxxxxx/

[2] 
https://lore.kernel.org/lkml/cover.1677672277.git.raghavendra.kt@xxxxxxx/T/#t

Thanks and Regards
- Raghu