Re: [RFC patch 7/7] [PATCH] glibc: nptl: Add support for attached pthread_mutexes

[Peter Zijlstra]

On Sat, Apr 02, 2016 at 06:30:59PM +0200, Peter Zijlstra wrote:
> On Sat, Apr 02, 2016 at 11:09:20AM -0000, Thomas Gleixner wrote:
> > pthread_mutexes on Linux are based on the futex mechanism. The standard futex
> > mechanism in the Linux kernel uses a global hash to store transient
> > state. Collisions on that hash can lead to performance degradation and on
> > real-time enabled kernels even to priority inversions.
> > 
> > To guarantee futexes without collisions on the global kernel hash, the kernel
> > provides a mechanism to attach to a futex. This creates futex private state
> > which avoids hash collisions and on NUMA systems also cross node memory
> > access.
> > 
> > To utilize this mechanism each thread has to attach to the futex before any
> > other operations on that futex which involve kernel interaction.
> > 
> > At pthread_mutex_init() the pthread_mutex attribute needs to be initialized
> > for attached mode via:
> > 
> >     pthread_mutexattr_setattached_np(&attr, 1);
> > 
> > All threads which are using the mutex - including the one which called
> > pthread_mutex_init() - must invoke
> > 
> >     pthread_mutex_attach_np(&mutex);
> > 
> > before any other pthread_mutex related operations.
> > 
> > Example:
> >         pthread_mutexattr_t attr;
> > 	pthread_mutex_t lock;
> > 
> >         pthread_mutexattr_init(&attr);
> >         pthread_mutexattr_setattached_np(&attr, 1);
> >         pthread_mutex_init(&lock, &attr);
> > 
> >         pthread_mutex_attach_np(&lock);
> > 
> > 	pthread_mutex_lock(&lock);
> > 
> > In ptrace this should look like this:
> > 
> > 	futex(<addr>, 0x280 /* FUTEX_??? */, 1, NULL <unfinished ...>
> > 
> > 	0x280: 	 FUTEX_ATTACHED | FUTEX_PRIVATE | FUTEX_WAIT
> > 
> > To undo the attachment each involved thread needs to call
> > 
> >     pthread_mutex_detach_np(&mutex);
> >         
> > When the last user detaches the kernel state is destroyed.
> 
> So I was fully expecting pthread_mutex_{at,de}tach_np() to not exist and
> be internal to pthread_mutex_{init,destroy}().
> 
> Is there a reason this is not so?

Ah yes there is; it just isn't spelled out and I'm tired.

This is because every thread that wants to play with the mutex needs to
attach, and only one can do init.