Re: RT patch acceptance

From: Philippe Gerum
Date: Tue May 24 2005 - 12:29:23 EST

Esben Nielsen wrote:
On Tue, 24 May 2005, Karim Yaghmour wrote:


Esben Nielsen wrote:

I find that a bad approach:
1) You don't have RT in userspace.
2) You can't use Linux drivers for standeard hardware when you want it to
be part of your deterministic RT application.

Please have a look at RTAI/fusion. For the record, RTAI has been providing
hard-rt in standard Linux user-space for over 5 years now. With RTAI/Fusion
this gets even better as there isn't even a special API ...


The tests I have read (I can't remember the links, but it was on lwn.net)
states that the worst case latency is even worse than for a standeard 2.6
kernel!

You are likely talking about a benchmark conducted by Peter Laurich and published by Linuxdevices, I guess. In fact, the benchmark code was wrong, and has been amended by the author himself in a follow-up to the initial article:

http://www.linuxdevices.com/articles/AT3479098230.html

The figures obtained with RTAI's LXRT extension are now correct, and not surprisingly, LXRT is the best performer among the solutions aimed at providing RT support in user-space in this study, with 48 us worst-case scheduling latency under high load on a SBC based on a Celeron running at 650 MHz, and populated with 512 MB SDRAM. PREEMPT_RT was not tested by the benchmark, though.


If you gonna make usefull deterministic real-time in userspace you got to
change stuff in kernel space and implement stuff like priority
inheritance, priority ceiling or similar. It can only turn up to be an
ugly hack which will end up being as intruesive into the kernel as Ingo's
approach. If you don't do anything like that you can not use _any_ Linux
kernel resources from your RT processes even though you have reimplemented
the pthread library to know about the "super RT" priorities.


Indeed, this is why the RTAI project has an experimental branch called "fusion", distinct from the classic LXRT one, which aims at a better integration of the real-time services it provides into the common kernel framework. The idea is to allow RTAI applications to be alternatively controlled by the Linux kernel and the real-time nucleus, while keeping the RT priority scheme intact across seamless and automatic transitions between both.

If you think that PREEMPT_RT will be able to give you real-time guarantees that are as stringent as those already exhibited by additional nucleus/co-schedulers on any kind of hardware including the embedded ones, then you will likely conclude that we are currently chasing wild gooses (and that we owe you one of them for Xmas). If not, well, maybe considering a symbiotic approach between a fully preemptable Linux kernel providing regular services and a specialized co-scheduler providing extreme predictability should make some sense, given that both co-operate to control a single set of real-time tasks in user-space.

But I give you: You will gain better interrupt latencies because
interrupts are executed below the Linux proper. I.e. when the Linux
kernel runs with interrupt disabled, they are really enabled in the RTAI
subsystem.


IIRC, interrupt latency has never been the toughest problem with the vanilla Linux kernel with respect to predictability, but the scheduling latency still is. Hence Ingo's work, not to speak of previous efforts regarding preemptability, I guess.

My estimate is that RTAI is good when you have a very small subsystem you
need to run RT with very low latencies. Forinstance, controlling a fast
device with limiting hardware resources to buffer events. For large control systems I don't think it is the proper way to do it.
There it is much better to run the control tasks as normal Linux
user-space processes with RT-priority. I can see Ingo's kernel doing that,
I can't see RTAI doing it except for very special situations where you
don't make _any_ Linux system calls at all! You can't even use a
normal Linux network device or character device from your RT application!



It happens that many if not most among the complex real-time applications have varying requirements with respect to determinism, depending on the task or execution context you consider from them. This is why the split application model, involving some kernel modules which implement the demanding RT stuff and some user-space programs connected to them, has been used for years.

What the RTAI project is trying to do now with its fusion branch, is to make compatible a larger spectrum of RT requirements without killing the design of your RT application as above. E.g. some tasks need to run on a 10Khz period, whilst others can deal with ~100 us jitter under high load , just for the purpose of being able to call regular Linux services; but you want all of them running embodied in a single regular user-space process, and being able to use GDB for chasing the gremlins in there.

For us, this implies to make the most preemptable Linux kernel we can find and the fusion nucleus share the same semantics and co-operate, instead of blindly running side-by-side, so that RTAI eventually appears as a native support provided by the Linux kernel to the real-time application designers. Sometimes, this requires RTAI to impersonate some vanilla system calls such as nanosleep(), so that you really have micro-second level wakeups, with a little help of RTAI's integrated oneshot timer. This also requires a bunch of headaches, coffee, and machines going south, but that's nothing worth documenting in a README, I guess.

--

Philippe.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/