Re: [RFC][PATCH] make global bitlock waitqueues per-node

[Linus Torvalds]

On Wed, Dec 21, 2016 at 4:30 AM, Nicholas Piggin <npiggin@xxxxxxxxx> wrote:
>
> I've been doing a bit of testing, and I don't know why you're seeing
> this.
>
> I don't think I've been able to trigger any actual page lock contention
> so nothing gets put on the waitqueue to really bounce cache lines around
> that I can see.

The "test is the waitqueue is empty" is going to cause cache misses
even if there is no contention.

In fact, that's why I want the contention bit in the struct page - not
because of any NUMA issues, but simply due to cache misses.

And yes, with no contention the bit waiting should hopefully be able
to cache things shared - which should make the bouncing much less -
but there's going to be a shitload of false sharing with any actual
IO, so you will get bouncing due to that.

And then regular bouncing due simply to capacity misses (rather than
the CPU's wanting exclusive access).

With the contention bit in place, the only people actually looking at
the wait queues are the ones doing IO. At which point false sharing is
going to go down dramatically, but even if it were to happen it goes
from a "big issue" to "who cares, the cachemiss is not noticeable
compared to the IO, even with a fast SSD".

                Linus