Definitely.
> Does NSPR create a pre-allocated
> pool of threads? (or some kind of adaptive pool?) If it's creating threads
> heavily (say per-request), then thats bad, at least with the current
> LinuxThreads implementation.
In one application, I have a thin layer over LinuxThreads which
provides a pool of threads. It makes a big difference (though the app
is deliberately an extreme case).
That said, IMHO the "pool of threads" approach on Linux is
fundamentally wrong, since (according to my measurements) the time
taken by a context switch tends to be more than half the time taken by
a clone(), thus it should be better to spawn threads directly. Of
course, all thread libraries will add *some* overheads on top of a
bare clone(), however...
> We have a 1:5 gap between the latency of clone() and
> pthread_create() there...
There are two clear sources of overhead in the current implementation
of pthread_create:
Firstly, it is currently implemented as:
1. Context switch to the manager thread.
2. The manager thread does the clone().
3. Context switch back to the calling thread.
One reason for this is that having the calling thread do the clone()
is much more complicated to implement (many potential race
conditions). Also, since a thread can exit before threads it created,
then these threads will be inherited by init. This causes two
problems.
1. If one of these orphaned threads dies abruptly (someone SIGKILLs
it, for instance) then the manager has AFAIK no way to find out that
the thread has died. This can result in the manager later trying to
send a signal to that thread (when it wants the whole "process" to
exit). This could have potentially unpleasant consequences if the PIDs
wrap (and heavy use of threads can churn through PIDs at a fearsome
rate). Such cases are pathological, but ought to be dealt with
robustly - they're a symptom of something missing from the current
model.
2. The times of the thread never contribute to the times of the
initial thread. I'm not sure how times ought to work, but currently
LinuxThreads tries to get all threads to contribute to the child times
of the initial thread.
Both of these problem could be neatly solved if the kernel
knew that thread-like processes live in teams. There are other
motivations for this: reasonable ps output, and robust gdb thread
support.
The second cause of overhead for pthread_create is much simpler: the
manager thread does a mmap when creating every thread for the
stack. (I've not measured the actual cost of this, it could be low
enough to not be worth worrying about). This could be avoided by
maintaining a pool of stacks (when a thread exits, it could use mmap
to "wipe" its stack, rather than munmaping it), with the manager
cleaning up unused stacks every so often.
-- Dave Wragg
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu