Re: clone() and pthreads?

Linus Torvalds (torvalds@cs.helsinki.fi)
Thu, 2 May 1996 08:41:14 +0300 (EET DST)


On Wed, 1 May 1996, Thomas =?ISO-8859-1?Q?K=F6nig?= wrote:
>
> what's clone() doing these days? One guy claimed (via E-Mail) that
> he'd written a clone()-based pthreads package, but didn't answer
> when I asked him where I could get it for testing.
>
> Does anybody know anything about this? Is clone() stable enough
> for this kind of thing?

Essentially, development on clone() stopped, because nobody was using it
and I couldn't get people to even try, and send me reports. I sent out a
silly test-program (which has shown up a couple times since then) that
should give technical people a starting point - certainly if somebody
would know enough to implement pthreads some other way they should have
no problem understanding that small code-fragment.

Essentially, the current status of clone() is that the code does work,
and it's used extensively inside the kernel itself, both for kernel
threads and for the normal "fork()" system call. As such, it should be
stable enough to do some real work on.

HOWEVER, it's not 100% completed, and I won't really be working on it
unless somebody actually starts to use it for some real project. There
are two notable problems with the current implementation:

- file descriptor locking. Currently, if you share file descriptors
among multiple processes (CLONE_FILES), and one process does a close()
on a file descriptor while another process is using that descriptor
for a read(), for example, the kernel can get rather upset.

Fixing this is trivial - the file descriptors already have the usage
counters required to keep track of this, it's just that they aren't
used for this kind of thing yet because with the traditional fork()
semantics this case can't happen. The patches would be scattered over
a number of files, but they would be otherwise pretty trivial.

- VM area locking. This is essentially the same as above, except it's a
case of one process unmapping a vm area while it's being used by
another process. Again the kernel doesn't test for that, and doesn't
like it.

Fixing this is not as trivial: there are many ways to handle this, and
there is no clear-cut "best way". It's not fundamentally hard, but it
requires some thought.

Anyway, both the problems are of the type "test for things that a
well-behaved program shouldn't be doing", so they should not impact good
processes, and as such the current implementation of clone() should be
good enough for testing. That's why I haven't bothered about it lately:
the fact that nobody has shown more than a lukewarm theoretical interest
means that I haven't felt that there was any reason to concentrate on
something that people don't really seem to be interested in.

I still feel that "clone()" is the correct interface to use, and while a
full pthreads library would require user support _too_, clone() would
give the possibility to run the program on multiple CPU's in a SMP
environment, and also allows the pthreads library to do truly
asynchronous operations (instead of using a select() loop). But I'll
readily admit that using clone() is not trivial, partly because threaded
programs (with "real" asynchronous threading) are not simple to write.

I'd be more than happy if somebody comes up with a real use for clone(),
and starts sending me problem reports.

Linus