Re: [RFC][PATCH 3/4] locking/mutex: Introduce __mutex_trylock_or_handoff()

[Waiman Long]

On 6/30/21 11:35 AM, Peter Zijlstra wrote:
> Yanfei reported that it is possible to loose HANDOFF when we race with
> mutex_unlock() and end up setting HANDOFF on an unlocked mutex. At
> that point anybody can steal it, loosing HANDOFF in the process.
>
> If this happens often enough, we can in fact starve the top waiter.
>
> Solve this by folding the 'set HANDOFF' operation into the trylock
> operation, such that either we acquire the lock, or it gets HANDOFF
> set. This avoids having HANDOFF set on an unlocked mutex.
>
> Reported-by: Yanfei Xu <yanfei.xu@xxxxxxxxxxxxx>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
> ---
>   kernel/locking/mutex.c |   58 +++++++++++++++++++++++++++++--------------------
>   1 file changed, 35 insertions(+), 23 deletions(-)
>
> --- a/kernel/locking/mutex.c
> +++ b/kernel/locking/mutex.c
> @@ -91,10 +91,7 @@ static inline unsigned long __owner_flag
>   	return owner & MUTEX_FLAGS;
>   }
>   
> -/*
> - * Trylock variant that returns the owning task on failure.
> - */
> -static inline struct task_struct *__mutex_trylock_or_owner(struct mutex *lock)
> +static inline struct task_struct *__mutex_trylock_common(struct mutex *lock, bool handoff)
>   {
>   	unsigned long owner, curr = (unsigned long)current;
>   
> @@ -104,39 +101,56 @@ static inline struct task_struct *__mute
>   		unsigned long task = owner & ~MUTEX_FLAGS;
>   
>   		if (task) {
> -			if (likely(task != curr))
> +			if (flags & MUTEX_FLAG_PICKUP) {
> +				if (task != curr)
> +					break;
> +				flags &= ~MUTEX_FLAG_HANDOFF;

I think you mean "flags &= ~MUTEX_FLAG_PICKUP". Right:-)

Cheers,
Longman