Re: [RFC PATCH] qspinlock: Improve performance by reducing load instruction rollback

From: Waiman Long
Date: Mon Oct 19 2015 - 13:21:00 EST


On 10/19/2015 05:33 AM, Peter Zijlstra wrote:
On Mon, Oct 19, 2015 at 10:27:22AM +0800, ling.ma.program@xxxxxxxxx wrote:
From: Ma Ling<ling.ml@xxxxxxxxxxxxxxx>

All load instructions can run speculatively but they have to follow
memory order rule in multiple cores as below:
_x = _y = 0

Processor 0 Processor 1

mov r1, [ _y] //M1 mov [ _x], 1 //M3
mov r2, [ _x] //M2 mov [ _y], 1 //M4

If r1 = 1, r2 must be 1

In order to guarantee above rule, although Processor 0 execute
M1 and M2 instruction out of order, they are kept in ROB,
when load buffer for _x in Processor 0 received the update
message from Processor 1, Processor 0 need to roll back
from M2 instruction, which will flush the whole pipeline,
the latency is over the penalty from branch prediction miss.

In this patch we use lock cmpxchg instruction to force load
instructions to be serialization, the destination operand
receives a write cycle without regard to the result of
the comparison, which can help us to reduce the penalty
from load instruction roll back.

Our experiment indicates the performance can be improved by 10%~15%
for 2 and 3 threads cases, the conflicts from lock cache line
spend them most of the time.
On what hardware? Also, you forgot to Cc Waiman, who is a prime author
of this code. Excessive quoting for his benefit.

Thanks for letting me aware of this patch. I had commented on the patch in a separate mail.

Cheers,
Longman

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/