Re: [LKP] [ext4] d3b6f23f71: stress-ng.fiemap.ops_per_sec -60.5% regression

[Xing Zhengjun]

Hi Harjani,

   Do you have time to take a look at this? Thanks.

On 4/7/2020 4:00 PM, kernel test robot wrote:
> Greeting,
>
> FYI, we noticed a -60.5% regression of stress-ng.fiemap.ops_per_sec due to commit:
>
>
> commit: d3b6f23f71670007817a5d59f3fbafab2b794e8c ("ext4: move ext4_fiemap to use iomap framework")
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>
> in testcase: stress-ng
> on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory
> with following parameters:
>
> 	nr_threads: 10%
> 	disk: 1HDD
> 	testtime: 1s
> 	class: os
> 	cpufreq_governor: performance
> 	ucode: 0x500002c
> 	fs: ext4
>
>
>
>
>
>
> Details are as below:
> -------------------------------------------------------------------------------------------------->
>
>
> To reproduce:
>
>          git clone https://github.com/intel/lkp-tests.git
>          cd lkp-tests
>          bin/lkp install job.yaml  # job file is attached in this email
>          bin/lkp run     job.yaml
>
> =========================================================================================
> class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode:
>    os/gcc-7/performance/1HDD/ext4/x86_64-rhel-7.6/10%/debian-x86_64-20191114.cgz/lkp-csl-2sp5/stress-ng/1s/0x500002c
>
> commit:
>    b2c5764262 ("ext4: make ext4_ind_map_blocks work with fiemap")
>    d3b6f23f71 ("ext4: move ext4_fiemap to use iomap framework")
>
> b2c5764262edded1 d3b6f23f71670007817a5d59f3f
> ---------------- ---------------------------
>         fail:runs  %reproduction    fail:runs
>             |             |             |
>             :4           25%           1:4     dmesg.WARNING:at#for_ip_interrupt_entry/0x
>            2:4            5%           2:4     perf-profile.calltrace.cycles-pp.sync_regs.error_entry
>            2:4            6%           3:4     perf-profile.calltrace.cycles-pp.error_entry
>            3:4            9%           3:4     perf-profile.children.cycles-pp.error_entry
>            0:4            1%           0:4     perf-profile.self.cycles-pp.error_entry
>           %stddev     %change         %stddev
>               \          |                \
>       28623           +28.2%      36703 ± 12%  stress-ng.daemon.ops
>       28632           +28.2%      36704 ± 12%  stress-ng.daemon.ops_per_sec
>      566.00 ± 22%     -53.2%     265.00 ± 53%  stress-ng.dev.ops
>      278.81 ± 22%     -53.0%     131.00 ± 54%  stress-ng.dev.ops_per_sec
>       73160           -60.6%      28849 ±  3%  stress-ng.fiemap.ops
>       72471           -60.5%      28612 ±  3%  stress-ng.fiemap.ops_per_sec
>       23421 ± 12%     +21.2%      28388 ±  6%  stress-ng.filename.ops
>       22638 ± 12%     +20.3%      27241 ±  6%  stress-ng.filename.ops_per_sec
>       21.25 ±  7%     -10.6%      19.00 ±  3%  stress-ng.iomix.ops
>       38.75 ± 49%     -47.7%      20.25 ± 96%  stress-ng.memhotplug.ops
>       34.45 ± 52%     -51.8%      16.62 ±106%  stress-ng.memhotplug.ops_per_sec
>        1734 ± 10%     +31.4%       2278 ± 10%  stress-ng.resources.ops
>      807.56 ±  5%     +35.2%       1091 ±  8%  stress-ng.resources.ops_per_sec
>     1007356 ±  3%     -16.5%     840642 ±  9%  stress-ng.revio.ops
>     1007692 ±  3%     -16.6%     840711 ±  9%  stress-ng.revio.ops_per_sec
>       21812 ±  3%     +16.0%      25294 ±  5%  stress-ng.sysbadaddr.ops
>       21821 ±  3%     +15.9%      25294 ±  5%  stress-ng.sysbadaddr.ops_per_sec
>      440.75 ±  4%     +21.9%     537.25 ±  9%  stress-ng.sysfs.ops
>      440.53 ±  4%     +21.9%     536.86 ±  9%  stress-ng.sysfs.ops_per_sec
>    13286582           -11.1%   11805520 ±  6%  stress-ng.time.file_system_outputs
>    68253896            +2.4%   69860122        stress-ng.time.minor_page_faults
>      197.00 ±  4%     -15.9%     165.75 ± 12%  stress-ng.xattr.ops
>      192.45 ±  5%     -16.1%     161.46 ± 11%  stress-ng.xattr.ops_per_sec
>       15310           +62.5%      24875 ± 22%  stress-ng.zombie.ops
>       15310           +62.5%      24874 ± 22%  stress-ng.zombie.ops_per_sec
>      203.50 ± 12%     -47.3%     107.25 ± 49%  vmstat.io.bi
>      861318 ± 18%     -29.7%     605884 ±  5%  meminfo.AnonHugePages
>     1062742 ± 14%     -20.2%     847853 ±  3%  meminfo.AnonPages
>       31093 ±  6%      +9.6%      34090 ±  3%  meminfo.KernelStack
>        7151 ± 34%     +55.8%      11145 ±  9%  meminfo.Mlocked
>   1.082e+08 ±  5%     -40.2%   64705429 ± 31%  numa-numastat.node0.local_node
>   1.082e+08 ±  5%     -40.2%   64739883 ± 31%  numa-numastat.node0.numa_hit
>    46032662 ± 21%    +104.3%   94042918 ± 20%  numa-numastat.node1.local_node
>    46074205 ± 21%    +104.2%   94072810 ± 20%  numa-numastat.node1.numa_hit
>        3942 ±  3%     +14.2%       4501 ±  4%  slabinfo.pool_workqueue.active_objs
>        4098 ±  3%     +14.3%       4683 ±  4%  slabinfo.pool_workqueue.num_objs
>        4817 ±  7%     +13.3%       5456 ±  8%  slabinfo.proc_dir_entry.active_objs
>        5153 ±  6%     +12.5%       5797 ±  8%  slabinfo.proc_dir_entry.num_objs
>       18598 ± 13%     -33.1%      12437 ± 20%  sched_debug.cfs_rq:/.load.avg
>      452595 ± 56%     -71.4%     129637 ± 76%  sched_debug.cfs_rq:/.load.max
>       67675 ± 35%     -55.1%      30377 ± 42%  sched_debug.cfs_rq:/.load.stddev
>       18114 ± 12%     -33.7%      12011 ± 20%  sched_debug.cfs_rq:/.runnable_weight.avg
>      448215 ± 58%     -72.8%     121789 ± 82%  sched_debug.cfs_rq:/.runnable_weight.max
>       67083 ± 37%     -56.3%      29305 ± 43%  sched_debug.cfs_rq:/.runnable_weight.stddev
>      -38032          +434.3%    -203212        sched_debug.cfs_rq:/.spread0.avg
>     -204466           +95.8%    -400301        sched_debug.cfs_rq:/.spread0.min
>       90.02 ± 25%     -58.1%      37.69 ± 52%  sched_debug.cfs_rq:/.util_est_enqueued.avg
>      677.54 ±  6%     -39.3%     411.50 ± 22%  sched_debug.cfs_rq:/.util_est_enqueued.max
>      196.57 ±  8%     -47.6%     103.05 ± 36%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
>        3.34 ± 23%     +34.1%       4.48 ±  4%  sched_debug.cpu.clock.stddev
>        3.34 ± 23%     +34.1%       4.48 ±  4%  sched_debug.cpu.clock_task.stddev
>      402872 ±  7%     -11.9%     354819 ±  2%  proc-vmstat.nr_active_anon
>     1730331            -9.5%    1566418 ±  5%  proc-vmstat.nr_dirtied
>       31042 ±  6%      +9.3%      33915 ±  3%  proc-vmstat.nr_kernel_stack
>      229047            -2.4%     223615        proc-vmstat.nr_mapped
>       74008 ±  7%     +20.5%      89163 ±  8%  proc-vmstat.nr_written
>      402872 ±  7%     -11.9%     354819 ±  2%  proc-vmstat.nr_zone_active_anon
>       50587 ± 11%     -25.2%      37829 ± 14%  proc-vmstat.numa_pages_migrated
>      457500           -23.1%     351918 ± 31%  proc-vmstat.numa_pte_updates
>    81382485            +1.9%   82907822        proc-vmstat.pgfault
>   2.885e+08 ±  5%     -13.3%  2.502e+08 ±  6%  proc-vmstat.pgfree
>       42206 ± 12%     -46.9%      22399 ± 49%  proc-vmstat.pgpgin
>      431233 ± 13%     -64.8%     151736 ±109%  proc-vmstat.pgrotated
>      176754 ±  7%     -40.2%     105637 ± 31%  proc-vmstat.thp_fault_alloc
>      314.50 ± 82%    +341.5%       1388 ± 44%  proc-vmstat.unevictable_pgs_stranded
>     1075269 ± 14%     -41.3%     631388 ± 17%  numa-meminfo.node0.Active
>      976056 ± 12%     -39.7%     588727 ± 19%  numa-meminfo.node0.Active(anon)
>      426857 ± 22%     -36.4%     271375 ± 13%  numa-meminfo.node0.AnonHugePages
>      558590 ± 19%     -36.4%     355402 ± 14%  numa-meminfo.node0.AnonPages
>     1794824 ±  9%     -28.8%    1277157 ± 20%  numa-meminfo.node0.FilePages
>        8517 ± 92%     -82.7%       1473 ± 89%  numa-meminfo.node0.Inactive(file)
>      633118 ±  2%     -41.7%     368920 ± 36%  numa-meminfo.node0.Mapped
>     2958038 ± 12%     -27.7%    2139271 ± 12%  numa-meminfo.node0.MemUsed
>      181401 ±  5%     -13.7%     156561 ±  4%  numa-meminfo.node0.SUnreclaim
>      258124 ±  6%     -13.0%     224535 ±  5%  numa-meminfo.node0.Slab
>      702083 ± 16%     +31.0%     919406 ± 11%  numa-meminfo.node1.Active
>       38663 ±107%    +137.8%      91951 ± 31%  numa-meminfo.node1.Active(file)
>     1154975 ±  7%     +41.6%    1635593 ± 12%  numa-meminfo.node1.FilePages
>      395813 ± 25%     +62.8%     644533 ± 16%  numa-meminfo.node1.Inactive
>      394313 ± 25%     +62.5%     640686 ± 16%  numa-meminfo.node1.Inactive(anon)
>      273317           +88.8%     515976 ± 25%  numa-meminfo.node1.Mapped
>     2279237 ±  6%     +25.7%    2865582 ±  7%  numa-meminfo.node1.MemUsed
>       10830 ± 18%     +29.6%      14033 ±  9%  numa-meminfo.node1.PageTables
>      149390 ±  3%     +23.2%     184085 ±  3%  numa-meminfo.node1.SUnreclaim
>      569542 ± 16%     +74.8%     995336 ± 21%  numa-meminfo.node1.Shmem
>      220774 ±  5%     +20.3%     265656 ±  3%  numa-meminfo.node1.Slab
>    35623587 ±  5%     -11.7%   31444514 ±  3%  perf-stat.i.cache-misses
>   2.576e+08 ±  5%      -6.8%    2.4e+08 ±  2%  perf-stat.i.cache-references
>        3585            -7.3%       3323 ±  5%  perf-stat.i.cpu-migrations
>      180139 ±  2%      +4.2%     187668        perf-stat.i.minor-faults
>       69.13            +2.6       71.75        perf-stat.i.node-load-miss-rate%
>     4313695 ±  2%      -7.4%    3994957 ±  2%  perf-stat.i.node-load-misses
>     5466253 ± 11%     -17.3%    4521173 ±  6%  perf-stat.i.node-loads
>     2818674 ±  6%     -15.8%    2372542 ±  5%  perf-stat.i.node-stores
>      227810            +4.6%     238290        perf-stat.i.page-faults
>       12.67 ±  4%      -7.2%      11.76 ±  2%  perf-stat.overall.MPKI
>        1.01 ±  4%      -0.0        0.97 ±  3%  perf-stat.overall.branch-miss-rate%
>        1044           +13.1%       1181 ±  4%  perf-stat.overall.cycles-between-cache-misses
>       40.37 ±  4%      +3.6       44.00 ±  2%  perf-stat.overall.node-store-miss-rate%
>    36139526 ±  5%     -12.5%   31625519 ±  3%  perf-stat.ps.cache-misses
>   2.566e+08 ±  5%      -6.9%  2.389e+08 ±  2%  perf-stat.ps.cache-references
>        3562            -7.2%       3306 ±  5%  perf-stat.ps.cpu-migrations
>      179088            +4.2%     186579        perf-stat.ps.minor-faults
>     4323383 ±  2%      -7.5%    3999214        perf-stat.ps.node-load-misses
>     5607721 ± 10%     -18.5%    4568664 ±  6%  perf-stat.ps.node-loads
>     2855134 ±  7%     -16.4%    2387345 ±  5%  perf-stat.ps.node-stores
>      226270            +4.6%     236709        perf-stat.ps.page-faults
>      242305 ± 10%     -42.4%     139551 ± 18%  numa-vmstat.node0.nr_active_anon
>      135983 ± 17%     -37.4%      85189 ± 10%  numa-vmstat.node0.nr_anon_pages
>      209.25 ± 16%     -38.1%     129.50 ± 10%  numa-vmstat.node0.nr_anon_transparent_hugepages
>      449367 ±  9%     -29.7%     315804 ± 20%  numa-vmstat.node0.nr_file_pages
>        2167 ± 90%     -80.6%     419.75 ± 98%  numa-vmstat.node0.nr_inactive_file
>      157405 ±  3%     -41.4%      92206 ± 35%  numa-vmstat.node0.nr_mapped
>        2022 ± 30%     -73.3%     539.25 ± 91%  numa-vmstat.node0.nr_mlock
>        3336 ± 10%     -24.3%       2524 ± 25%  numa-vmstat.node0.nr_page_table_pages
>      286158 ± 10%     -41.2%     168337 ± 37%  numa-vmstat.node0.nr_shmem
>       45493 ±  5%     -14.1%      39094 ±  4%  numa-vmstat.node0.nr_slab_unreclaimable
>      242294 ± 10%     -42.4%     139547 ± 18%  numa-vmstat.node0.nr_zone_active_anon
>        2167 ± 90%     -80.6%     419.75 ± 98%  numa-vmstat.node0.nr_zone_inactive_file
>    54053924 ±  8%     -39.3%   32786242 ± 34%  numa-vmstat.node0.numa_hit
>    53929628 ±  8%     -39.5%   32619715 ± 34%  numa-vmstat.node0.numa_local
>        9701 ±107%    +136.9%      22985 ± 31%  numa-vmstat.node1.nr_active_file
>      202.50 ± 16%     -25.1%     151.75 ± 23%  numa-vmstat.node1.nr_anon_transparent_hugepages
>      284922 ±  7%     +43.3%     408195 ± 13%  numa-vmstat.node1.nr_file_pages
>       96002 ± 26%     +67.5%     160850 ± 17%  numa-vmstat.node1.nr_inactive_anon
>       68077 ±  2%     +90.3%     129533 ± 25%  numa-vmstat.node1.nr_mapped
>      138482 ± 15%     +79.2%     248100 ± 22%  numa-vmstat.node1.nr_shmem
>       37396 ±  3%     +23.3%      46094 ±  3%  numa-vmstat.node1.nr_slab_unreclaimable
>        9701 ±107%    +136.9%      22985 ± 31%  numa-vmstat.node1.nr_zone_active_file
>       96005 ± 26%     +67.5%     160846 ± 17%  numa-vmstat.node1.nr_zone_inactive_anon
>    23343661 ± 17%     +99.9%   46664267 ± 23%  numa-vmstat.node1.numa_hit
>    23248487 ± 17%    +100.5%   46610447 ± 23%  numa-vmstat.node1.numa_local
>      105745 ± 23%    +112.6%     224805 ± 24%  softirqs.CPU0.NET_RX
>      133310 ± 36%     -45.3%      72987 ± 52%  softirqs.CPU1.NET_RX
>      170110 ± 55%     -66.8%      56407 ±147%  softirqs.CPU11.NET_RX
>       91465 ± 36%     -65.2%      31858 ±112%  softirqs.CPU13.NET_RX
>      164491 ± 57%     -77.7%      36641 ±121%  softirqs.CPU15.NET_RX
>      121069 ± 55%     -99.3%     816.75 ± 96%  softirqs.CPU17.NET_RX
>       81019 ±  4%      -8.7%      73967 ±  4%  softirqs.CPU20.RCU
>       72143 ± 63%     -89.8%       7360 ±172%  softirqs.CPU22.NET_RX
>      270663 ± 17%     -57.9%     113915 ± 45%  softirqs.CPU24.NET_RX
>       20149 ± 76%    +474.1%     115680 ± 62%  softirqs.CPU26.NET_RX
>       14033 ± 70%    +977.5%     151211 ± 75%  softirqs.CPU27.NET_RX
>       27834 ± 94%    +476.1%     160357 ± 28%  softirqs.CPU28.NET_RX
>       35346 ± 68%    +212.0%     110290 ± 30%  softirqs.CPU29.NET_RX
>       34347 ±103%    +336.5%     149941 ± 32%  softirqs.CPU32.NET_RX
>       70077 ±  3%     +10.8%      77624 ±  3%  softirqs.CPU34.RCU
>       36453 ± 84%    +339.6%     160253 ± 42%  softirqs.CPU36.NET_RX
>       72367 ±  2%     +10.6%      80043        softirqs.CPU37.RCU
>       25239 ±118%    +267.7%      92799 ± 45%  softirqs.CPU38.NET_RX
>        4995 ±170%   +1155.8%      62734 ± 62%  softirqs.CPU39.NET_RX
>        4641 ±145%   +1611.3%      79432 ± 90%  softirqs.CPU42.NET_RX
>        7192 ± 65%    +918.0%      73225 ± 66%  softirqs.CPU45.NET_RX
>        1772 ±166%   +1837.4%      34344 ± 63%  softirqs.CPU46.NET_RX
>       13149 ± 81%    +874.7%     128170 ± 58%  softirqs.CPU47.NET_RX
>       86484 ± 94%     -92.6%       6357 ±172%  softirqs.CPU48.NET_RX
>      129128 ± 27%     -95.8%       5434 ±172%  softirqs.CPU55.NET_RX
>       82772 ± 59%     -91.7%       6891 ±164%  softirqs.CPU56.NET_RX
>      145313 ± 57%     -87.8%      17796 ± 88%  softirqs.CPU57.NET_RX
>      118160 ± 33%     -86.3%      16226 ±109%  softirqs.CPU58.NET_RX
>       94576 ± 56%     -94.1%       5557 ±173%  softirqs.CPU6.NET_RX
>       82900 ± 77%     -66.8%      27508 ±171%  softirqs.CPU62.NET_RX
>      157291 ± 30%     -81.1%      29656 ±111%  softirqs.CPU64.NET_RX
>      135101 ± 28%     -80.2%      26748 ± 90%  softirqs.CPU67.NET_RX
>      146574 ± 56%    -100.0%      69.75 ± 98%  softirqs.CPU68.NET_RX
>       81347 ±  2%      -9.0%      74024 ±  2%  softirqs.CPU68.RCU
>      201729 ± 37%     -99.6%     887.50 ±107%  softirqs.CPU69.NET_RX
>      108454 ± 78%     -97.9%       2254 ±169%  softirqs.CPU70.NET_RX
>       55289 ±104%     -89.3%       5942 ±172%  softirqs.CPU71.NET_RX
>       10112 ±172%    +964.6%     107651 ± 89%  softirqs.CPU72.NET_RX
>        3136 ±171%   +1522.2%      50879 ± 66%  softirqs.CPU73.NET_RX
>       13353 ± 79%    +809.2%     121407 ±101%  softirqs.CPU74.NET_RX
>       75194 ±  3%     +10.3%      82957 ±  5%  softirqs.CPU75.RCU
>       11002 ±173%   +1040.8%     125512 ± 61%  softirqs.CPU76.NET_RX
>        2463 ±173%   +2567.3%      65708 ± 77%  softirqs.CPU78.NET_RX
>       25956 ±  3%      -7.8%      23932 ±  3%  softirqs.CPU78.SCHED
>       16366 ±150%    +340.7%      72125 ± 91%  softirqs.CPU82.NET_RX
>       14553 ±130%   +1513.4%     234809 ± 27%  softirqs.CPU93.NET_RX
>       26314            -9.2%      23884 ±  3%  softirqs.CPU93.SCHED
>        4582 ± 88%   +4903.4%     229268 ± 23%  softirqs.CPU94.NET_RX
>       11214 ±111%   +1762.5%     208867 ± 18%  softirqs.CPU95.NET_RX
>        1.53 ± 27%      -0.5        0.99 ± 17%  perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
>        1.52 ± 27%      -0.5        0.99 ± 17%  perf-profile.calltrace.cycles-pp.do_signal.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
>        1.39 ± 29%      -0.5        0.88 ± 21%  perf-profile.calltrace.cycles-pp.do_group_exit.get_signal.do_signal.exit_to_usermode_loop.do_syscall_64
>        1.39 ± 29%      -0.5        0.88 ± 21%  perf-profile.calltrace.cycles-pp.get_signal.do_signal.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
>        0.50 ± 59%      +0.3        0.81 ± 13%  perf-profile.calltrace.cycles-pp.filemap_map_pages.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault
>        5.70 ±  9%      +0.8        6.47 ±  7%  perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.get_signal.do_signal.exit_to_usermode_loop
>        5.48 ±  9%      +0.8        6.27 ±  7%  perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.get_signal
>        5.49 ±  9%      +0.8        6.28 ±  7%  perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.get_signal.do_signal
>        4.30 ±  4%      +1.3        5.60 ±  7%  perf-profile.calltrace.cycles-pp.do_group_exit.get_signal.do_signal.exit_to_usermode_loop.prepare_exit_to_usermode
>        4.40 ±  4%      +1.3        5.69 ±  7%  perf-profile.calltrace.cycles-pp.prepare_exit_to_usermode.swapgs_restore_regs_and_return_to_usermode
>        4.37 ±  4%      +1.3        5.66 ±  7%  perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.prepare_exit_to_usermode.swapgs_restore_regs_and_return_to_usermode
>        4.36 ±  4%      +1.3        5.66 ±  7%  perf-profile.calltrace.cycles-pp.do_signal.exit_to_usermode_loop.prepare_exit_to_usermode.swapgs_restore_regs_and_return_to_usermode
>        4.33 ±  4%      +1.3        5.62 ±  7%  perf-profile.calltrace.cycles-pp.get_signal.do_signal.exit_to_usermode_loop.prepare_exit_to_usermode.swapgs_restore_regs_and_return_to_usermode
>        4.44 ±  4%      +1.3        5.74 ±  7%  perf-profile.calltrace.cycles-pp.swapgs_restore_regs_and_return_to_usermode
>        3.20 ± 10%      -2.4        0.78 ±156%  perf-profile.children.cycles-pp.copy_page
>        0.16 ±  9%      -0.1        0.08 ± 64%  perf-profile.children.cycles-pp.irq_work_interrupt
>        0.16 ±  9%      -0.1        0.08 ± 64%  perf-profile.children.cycles-pp.smp_irq_work_interrupt
>        0.24 ±  5%      -0.1        0.17 ± 18%  perf-profile.children.cycles-pp.irq_work_run_list
>        0.16 ±  9%      -0.1        0.10 ± 24%  perf-profile.children.cycles-pp.irq_work_run
>        0.16 ±  9%      -0.1        0.10 ± 24%  perf-profile.children.cycles-pp.printk
>        0.23 ±  6%      -0.1        0.17 ±  9%  perf-profile.children.cycles-pp.__do_execve_file
>        0.08 ± 14%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.delay_tsc
>        0.16 ±  6%      -0.1        0.11 ±  9%  perf-profile.children.cycles-pp.load_elf_binary
>        0.16 ±  7%      -0.0        0.12 ± 13%  perf-profile.children.cycles-pp.search_binary_handler
>        0.20 ±  7%      -0.0        0.15 ± 10%  perf-profile.children.cycles-pp.call_usermodehelper_exec_async
>        0.19 ±  6%      -0.0        0.15 ± 11%  perf-profile.children.cycles-pp.do_execve
>        0.08 ± 10%      -0.0        0.04 ± 59%  perf-profile.children.cycles-pp.__vunmap
>        0.15 ±  3%      -0.0        0.11 ±  7%  perf-profile.children.cycles-pp.rcu_idle_exit
>        0.12 ± 10%      -0.0        0.09 ± 14%  perf-profile.children.cycles-pp.__switch_to_asm
>        0.09 ± 13%      -0.0        0.07 ±  5%  perf-profile.children.cycles-pp.des3_ede_encrypt
>        0.06 ± 11%      +0.0        0.09 ± 13%  perf-profile.children.cycles-pp.mark_page_accessed
>        0.15 ±  5%      +0.0        0.19 ± 12%  perf-profile.children.cycles-pp.apparmor_cred_prepare
>        0.22 ±  8%      +0.0        0.27 ± 11%  perf-profile.children.cycles-pp.mem_cgroup_throttle_swaprate
>        0.17 ±  2%      +0.0        0.22 ± 12%  perf-profile.children.cycles-pp.security_prepare_creds
>        0.95 ± 17%      +0.3        1.22 ± 14%  perf-profile.children.cycles-pp.filemap_map_pages
>        5.92 ±  8%      +0.7        6.65 ±  7%  perf-profile.children.cycles-pp.get_signal
>        5.66 ±  9%      +0.8        6.44 ±  7%  perf-profile.children.cycles-pp.mmput
>        5.65 ±  9%      +0.8        6.43 ±  7%  perf-profile.children.cycles-pp.exit_mmap
>        4.40 ±  4%      +1.3        5.70 ±  7%  perf-profile.children.cycles-pp.prepare_exit_to_usermode
>        4.45 ±  4%      +1.3        5.75 ±  7%  perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
>        3.16 ± 10%      -2.4        0.77 ±155%  perf-profile.self.cycles-pp.copy_page
>        0.08 ± 14%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.delay_tsc
>        0.12 ± 10%      -0.0        0.09 ± 14%  perf-profile.self.cycles-pp.__switch_to_asm
>        0.08 ± 12%      -0.0        0.06 ± 17%  perf-profile.self.cycles-pp.enqueue_task_fair
>        0.09 ± 13%      -0.0        0.07 ±  5%  perf-profile.self.cycles-pp.des3_ede_encrypt
>        0.07 ± 13%      +0.0        0.08 ± 19%  perf-profile.self.cycles-pp.__lru_cache_add
>        0.19 ±  9%      +0.0        0.22 ± 10%  perf-profile.self.cycles-pp.mem_cgroup_throttle_swaprate
>        0.15 ±  5%      +0.0        0.19 ± 11%  perf-profile.self.cycles-pp.apparmor_cred_prepare
>        0.05 ± 58%      +0.0        0.09 ± 13%  perf-profile.self.cycles-pp.mark_page_accessed
>        0.58 ± 10%      +0.2        0.80 ± 20%  perf-profile.self.cycles-pp.release_pages
>        0.75 ±173%  +1.3e+05%       1005 ±100%  interrupts.127:PCI-MSI.31981660-edge.i40e-eth0-TxRx-91
>      820.75 ±111%     -99.9%       0.50 ±173%  interrupts.47:PCI-MSI.31981580-edge.i40e-eth0-TxRx-11
>      449.25 ± 86%    -100.0%       0.00        interrupts.53:PCI-MSI.31981586-edge.i40e-eth0-TxRx-17
>       33.25 ±157%    -100.0%       0.00        interrupts.57:PCI-MSI.31981590-edge.i40e-eth0-TxRx-21
>        0.75 ±110%  +63533.3%     477.25 ±162%  interrupts.61:PCI-MSI.31981594-edge.i40e-eth0-TxRx-25
>      561.50 ±160%    -100.0%       0.00        interrupts.65:PCI-MSI.31981598-edge.i40e-eth0-TxRx-29
>       82921 ±  8%     -11.1%      73748 ±  6%  interrupts.CPU11.CAL:Function_call_interrupts
>       66509 ± 30%     -32.6%      44828 ±  8%  interrupts.CPU14.TLB:TLB_shootdowns
>       43105 ± 98%     -90.3%       4183 ± 21%  interrupts.CPU17.RES:Rescheduling_interrupts
>      148719 ± 70%     -69.4%      45471 ± 16%  interrupts.CPU17.TLB:TLB_shootdowns
>       85589 ± 42%     -52.2%      40884 ±  5%  interrupts.CPU20.TLB:TLB_shootdowns
>      222472 ± 41%     -98.0%       4360 ± 45%  interrupts.CPU22.RES:Rescheduling_interrupts
>        0.50 ±173%  +95350.0%     477.25 ±162%  interrupts.CPU25.61:PCI-MSI.31981594-edge.i40e-eth0-TxRx-25
>       76029 ± 10%     +14.9%      87389 ±  5%  interrupts.CPU25.CAL:Function_call_interrupts
>      399042 ±  6%     +13.4%     452479 ±  8%  interrupts.CPU27.LOC:Local_timer_interrupts
>      561.00 ±161%    -100.0%       0.00        interrupts.CPU29.65:PCI-MSI.31981598-edge.i40e-eth0-TxRx-29
>        7034 ± 46%   +1083.8%      83279 ±138%  interrupts.CPU29.RES:Rescheduling_interrupts
>       17829 ± 99%     -71.0%       5172 ± 16%  interrupts.CPU30.RES:Rescheduling_interrupts
>        5569 ± 15%   +2414.7%     140059 ± 94%  interrupts.CPU31.RES:Rescheduling_interrupts
>       37674 ± 16%     +36.6%      51473 ± 25%  interrupts.CPU31.TLB:TLB_shootdowns
>       47905 ± 39%     +76.6%      84583 ± 38%  interrupts.CPU34.TLB:TLB_shootdowns
>      568.75 ±140%    +224.8%       1847 ± 90%  interrupts.CPU36.NMI:Non-maskable_interrupts
>      568.75 ±140%    +224.8%       1847 ± 90%  interrupts.CPU36.PMI:Performance_monitoring_interrupts
>        4236 ± 25%   +2168.5%      96092 ± 90%  interrupts.CPU36.RES:Rescheduling_interrupts
>       52717 ± 27%     +43.3%      75565 ± 28%  interrupts.CPU37.TLB:TLB_shootdowns
>       41418 ±  9%    +136.6%      98010 ± 50%  interrupts.CPU39.TLB:TLB_shootdowns
>        5551 ±  8%    +847.8%      52615 ± 66%  interrupts.CPU40.RES:Rescheduling_interrupts
>        4746 ± 25%    +865.9%      45841 ± 91%  interrupts.CPU42.RES:Rescheduling_interrupts
>       37556 ± 11%     +24.6%      46808 ±  6%  interrupts.CPU42.TLB:TLB_shootdowns
>       21846 ±124%     -84.4%       3415 ± 46%  interrupts.CPU48.RES:Rescheduling_interrupts
>      891.50 ± 22%     -35.2%     577.25 ± 40%  interrupts.CPU49.NMI:Non-maskable_interrupts
>      891.50 ± 22%     -35.2%     577.25 ± 40%  interrupts.CPU49.PMI:Performance_monitoring_interrupts
>       20459 ±120%     -79.2%       4263 ± 14%  interrupts.CPU49.RES:Rescheduling_interrupts
>       59840 ± 21%     -23.1%      46042 ± 16%  interrupts.CPU5.TLB:TLB_shootdowns
>       65200 ± 19%     -34.5%      42678 ±  9%  interrupts.CPU51.TLB:TLB_shootdowns
>       70923 ±153%     -94.0%       4270 ± 29%  interrupts.CPU53.RES:Rescheduling_interrupts
>       65312 ± 22%     -28.7%      46578 ± 14%  interrupts.CPU56.TLB:TLB_shootdowns
>       65828 ± 24%     -33.4%      43846 ±  4%  interrupts.CPU59.TLB:TLB_shootdowns
>       72558 ±156%     -93.2%       4906 ±  9%  interrupts.CPU6.RES:Rescheduling_interrupts
>       68698 ± 34%     -32.6%      46327 ± 18%  interrupts.CPU61.TLB:TLB_shootdowns
>      109745 ± 44%     -57.4%      46711 ± 16%  interrupts.CPU62.TLB:TLB_shootdowns
>       89714 ± 44%     -48.5%      46198 ±  7%  interrupts.CPU63.TLB:TLB_shootdowns
>       59380 ±136%     -91.5%       5066 ± 13%  interrupts.CPU69.RES:Rescheduling_interrupts
>       40094 ± 18%    +133.9%      93798 ± 44%  interrupts.CPU78.TLB:TLB_shootdowns
>      129884 ± 72%     -55.3%      58034 ±157%  interrupts.CPU8.RES:Rescheduling_interrupts
>       69984 ± 11%     +51.4%     105957 ± 20%  interrupts.CPU80.CAL:Function_call_interrupts
>       32857 ± 10%    +128.7%      75131 ± 36%  interrupts.CPU80.TLB:TLB_shootdowns
>       35726 ± 16%     +34.6%      48081 ± 12%  interrupts.CPU82.TLB:TLB_shootdowns
>       73820 ± 17%     +28.2%      94643 ±  8%  interrupts.CPU84.CAL:Function_call_interrupts
>       38829 ± 28%    +190.3%     112736 ± 42%  interrupts.CPU84.TLB:TLB_shootdowns
>       36129 ±  4%     +47.6%      53329 ± 13%  interrupts.CPU85.TLB:TLB_shootdowns
>        4693 ±  7%   +1323.0%      66793 ±145%  interrupts.CPU86.RES:Rescheduling_interrupts
>       38003 ± 11%     +94.8%      74031 ± 43%  interrupts.CPU86.TLB:TLB_shootdowns
>       78022 ±  3%      +7.9%      84210 ±  3%  interrupts.CPU87.CAL:Function_call_interrupts
>       36359 ±  6%     +54.9%      56304 ± 48%  interrupts.CPU88.TLB:TLB_shootdowns
>       89031 ±105%     -95.0%       4475 ± 40%  interrupts.CPU9.RES:Rescheduling_interrupts
>       40085 ± 11%     +60.6%      64368 ± 27%  interrupts.CPU91.TLB:TLB_shootdowns
>       42244 ± 10%     +44.8%      61162 ± 35%  interrupts.CPU94.TLB:TLB_shootdowns
>       40959 ± 15%    +109.4%      85780 ± 41%  interrupts.CPU95.TLB:TLB_shootdowns
>
>
>                                                                                  
>                                  stress-ng.fiemap.ops
>                                                                                  
>    80000 +-------------------------------------------------------------------+
>    75000 |..+.             .+..            .+..+..  .+.  .+..                |
>          |    +..+..+..+.+.     .+..+..  .+       +.   +.    +.+..+..+..+.+..|
>    70000 |-+                   +       +.                                    |
>    65000 |-+                                                                 |
>    60000 |-+                                                                 |
>    55000 |-+                                                                 |
>          |                                                                   |
>    50000 |-+                                                                 |
>    45000 |-+                                                                 |
>    40000 |-+                                                                 |
>    35000 |-+                O                                                |
>          |  O       O                       O     O    O                     |
>    30000 |-+  O  O     O O     O O     O  O    O     O    O  O O  O  O  O O  |
>    25000 +-------------------------------------------------------------------+
>                                                                                  
>                                                                                                                                                                  
>                              stress-ng.fiemap.ops_per_sec
>                                                                                  
>    80000 +-------------------------------------------------------------------+
>    75000 |..               .+..               .+..                           |
>          |  +.  .+..+..+.+.     .+..+..  .+.+.    +..+.+..+..+.+..+..+..+.+..|
>    70000 |-+  +.               +       +.                                    |
>    65000 |-+                                                                 |
>    60000 |-+                                                                 |
>    55000 |-+                                                                 |
>          |                                                                   |
>    50000 |-+                                                                 |
>    45000 |-+                                                                 |
>    40000 |-+                                                                 |
>    35000 |-+                O                                                |
>          |  O       O                       O     O                          |
>    30000 |-+  O  O     O O     O O     O       O     O O  O  O    O  O  O O  |
>    25000 +-------------------------------------------------------------------+
>                                                                                  
>                                                                                  
> [*] bisect-good sample
> [O] bisect-bad  sample
>
>
>
> Disclaimer:
> Results have been estimated based on internal Intel analysis and are provided
> for informational purposes only. Any difference in system hardware or software
> design or configuration may affect actual performance.
>
>
> Thanks,
> Rong Chen
>
>
> _______________________________________________
> LKP mailing list -- lkp@xxxxxxxxxxxx
> To unsubscribe send an email to lkp-leave@xxxxxxxxxxxx

--
Zhengjun Xing