Re: [PATCH v6 5/6] locking/pvqspinlock: Allow 1 lock stealing attempt

[Waiman Long]

On 09/14/2015 10:00 AM, Peter Zijlstra wrote:
> On Fri, Sep 11, 2015 at 02:37:37PM -0400, Waiman Long wrote:
> > This patch allows one attempt for the lock waiter to steal the lock
> > when entering the PV slowpath.  This helps to reduce the performance
> > penalty caused by lock waiter preemption while not having much of
> > the downsides of a real unfair lock.
> > @@ -415,8 +458,12 @@ static void pv_wait_head(struct qspinlock *lock, struct mcs_spinlock *node)
> >
> >   	for (;; waitcnt++) {
> >   		for (loop = SPIN_THRESHOLD; loop; loop--) {
> > -			if (!READ_ONCE(l->locked))
> > -				return;
> > +			/*
> > +			 * Try to acquire the lock when it is free.
> > +			 */
> > +			if (!READ_ONCE(l->locked)&&
> > +			   (cmpxchg(&l->locked, 0, _Q_LOCKED_VAL) == 0))
> > +				goto gotlock;
> >   			cpu_relax();
> >   		}
> This isn't _once_, this is once per 'wakeup'. And note that interrupts
> unrelated to the kick can equally wake the vCPU up.

Oh! There is a minor bug that I shouldn't need to have a second 
READ_ONCE() call here.

As this is the queue head, finding the lock free entitles the vCPU to 
own the lock. However, because of lock stealing, I can't just write a 1 
to the lock and assume thing is all set. That is why I need to use 
cmpxchg() to make sure that the queue head vCPU can actually get the 
lock without the lock stolen underneath. I don't count that as lock 
stealing as it is the rightful owner of the lock.

I am sorry that I should have added a comment to clarify that. Will do 
so in the next update.

> void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
> {
>     :
>         /*
>         * We touched a (possibly) cold cacheline in the per-cpu queue 
node;
>         * attempt the trylock once more in the hope someone let go 
while we
>         * weren't watching.
>         */
>        if (queued_spin_trylock(lock))
>                goto release;

This is the only place where I consider lock stealing happens. Again, I 
should have a comment in pv_queued_spin_trylock_unfair() to say where it 
will be called.

Cheers,
Longman
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/