Re: [mm/writeback] 8d92890bd6: will-it-scale.per_process_ops -15.3% regression

[Jan Kara]

On Thu 15-10-20 08:46:01, NeilBrown wrote:
> On Wed, Oct 14 2020, Jan Kara wrote:
> 
> > On Wed 14-10-20 16:47:06, kernel test robot wrote:
> >> Greeting,
> >> 
> >> FYI, we noticed a -15.3% regression of will-it-scale.per_process_ops due
> >> to commit:
> >> 
> >> commit: 8d92890bd6b8502d6aee4b37430ae6444ade7a8c ("mm/writeback: discard
> >> NR_UNSTABLE_NFS, use NR_WRITEBACK instead")
> >> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> >
> > Thanks for report but it doesn't quite make sense to me. If we omit
> > reporting & NFS changes in that commit (which is code not excercised by
> > this benchmark), what remains are changes like:
> >
> >         nr_pages += node_page_state(pgdat, NR_FILE_DIRTY);
> > -       nr_pages += node_page_state(pgdat, NR_UNSTABLE_NFS);
> >         nr_pages += node_page_state(pgdat, NR_WRITEBACK);
> > ...
> > -               nr_reclaimable = global_node_page_state(NR_FILE_DIRTY) +
> > -                                       global_node_page_state(NR_UNSTABLE_NFS);
> > +               nr_reclaimable = global_node_page_state(NR_FILE_DIRTY);
> > ...
> > -       gdtc->dirty = global_node_page_state(NR_FILE_DIRTY) +
> > -                     global_node_page_state(NR_UNSTABLE_NFS);
> > +       gdtc->dirty = global_node_page_state(NR_FILE_DIRTY);
> >
> > So if there's any negative performance impact of these changes, they're
> > likely due to code alignment changes or something like that... So I don't
> > think there's much to do here since optimal code alignment is highly specific
> > to a particular CPU etc.
> 
> I agree, it seems odd.
> 
> Removing NR_UNSTABLE_NFS from enum node_stat_item would renumber all the
> following value and would, I think, change NR_DIRTIED from 32 to 31.
> Might that move something to a different cache line and change some
> contention?

Interesting theory, it could be possible.

> That would be easy enough to test: just re-add NR_UNSTABLE_NFS.

Yeah, easy enough to test. Patch for this is attached. 0-day people, can
you check whether applying this patch changes anything in your perf
numbers?

> I have no experience reading will-it-scale results, but 15% does seem
> like a lot.

Well, will-it-scale is a micro-benchmark that usually runs in extremely
parallel loads so 15% can be caused by fairly obscure issues like different
code alignment of a hot loop, slightly different cache line sharing, or
so...

								Honza
-- 
Jan Kara <jack@xxxxxxxx>
SUSE Labs, CR