Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

From: Michael K. Edwards
Date: Wed Feb 21 2007 - 19:53:28 EST

On 2/21/07, Ingo Molnar <mingo@xxxxxxx> wrote:
threadlets, when they dont block, are just regular user-space function
calls - so no need to schedule or throttle them. [*]

Right. That's a great design feature.

threadlets, when they block, are regular kernel threads, so the regular
O(1) scheduler takes care of them. If MMU trashing is of any concern
then syslets should be used to implement the most performance-critical
events: under Linux a kernel thread that does not exit out to user-space
does not do any TLB switching at all. (even if there are multiple
processes active and their syslets intermix)

As far as I am concerned syslets by themselves are a dead letter,
because you can't do any of the things that potential application
coders need to do with them. As for threadlets, making them kernel
threads is not such a good design feature, O(1) scheduler or not. You
take the full hit of kernel task creation, on the spot, for every
threadlet that blocks. You don't fence off the threadlet from any of
the stuff that it ought to be fenced off from for thread-safety
reasons, so you don't have much choice but to create a new TLS arena
for it (which you need anyway for errno), so they have horrible MMU
and memory overhead. You can't dispatch them inexpensively, while the
data delivered by a softirq is still hot in cache, to traverse 1-3
lines of userspace code and make the next syscall. So they're just
not usable for any of the things that a real AIO application actually

throttling of outstanding async contexts is most easily done by
user-space - you can see an example in threadlet-test.c, but there's
also fio/engines/syslet-rw.c. v2 had a kernel-space throttling mechanism
as well, i'll probably reintroduce that in later versions.

You're telling me that scheduling parallel I/O is the kernel's job but
throttling it is userspace's job?

[*] although certain more advanced scheduling tactics like the detection
of frequently executed threadlet functions and their pushing out to
separate contexts is possible too - but this is an optional add-on
and for later.

You won't be able to do it later if you don't design for it now.
Don't reinvent the square wheel -- there's a model to follow that was
so successful that it has killed all alternate models in its sphere.
Namely, IEEE 754. But please try not to make pipeline flushes suck as
much as they did on the i860.

- Michael
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at