> > I'm not sure what you mean by 'sources' here. Different
> > manufacturers? Different types of video I/O?
>
> Different data sources.
Oh, OK. Yes, this is most definitely an issue, which is why
EvStack (The GGI project's new console subsystem replacement) is based on
a message-passing paradigm similar to what you commonly see with
networking protocols. As long as the subject has come up (hehehe), let me
take this opportunity to expound in more detail upon the wonderfulness
that is EvStack. The following description/lecture is very long, but as
it pertains to what the GGI project feels is a very fundamental revolution
in a very fundamental core component of the Linux kernel, I cannot do it
justice without a detailed explanation.
Also, keep in mind that, while I am a GGI developer and have
played around quite a bit with what is currently available, EvStack is
very much a work in progress and the final authority on its minutiae is
Andreas Beck, founder of the GGI project. If you have any further
questions/suggestions/complaints about EvStack, he is the one to talk to.
EvStack is one of the most innovative ideas I have come across in quite
some time (though perhaps not as much to some on this list with more OS
design experience than myself), and my hat is definitely off to Andy.
With all that in mind, here we go....
There are a lot of similarities between console I/O and network
I/O: both deal with very high bandwidth data streams, both have to handle
data flowing from many different sources to many different destinations
with potentially complex multi-level routing and protocols, both have to
handle many different types of data in a generic, flexible and fast
manner, and both benefit from tight kernel integration due to their
necessarily tight hardware coupling. EvStack is based upon this concept,
which reduces all console I/O handling to the routing and tweaking of
streams of messages and packetized data passing from input sources of
widely varying types to output devices, also of widely varying types.
When you are designing a system for handling as many divergent
types of input and output devices as exist on the whole range of computing
hardware available today (which the GGI has to, because Linux will
eventually run on everything), this type of message passing system is
pretty much a necessity. You simply cannot provide an adequate level of
speed and flexibility with a traditional function-based API - even if you
extend one of those to cover every single I/O device currently in
existence, it'll become a bloated monstrosity, which will become even more
bloated over time as more and more new devices need supporting, AND it
will be changing out from under the developers every day!. It just will
not work. Message passing, on the other hand, is almost infinitely
flexible and extensible. All you have to do to see this is look at any
networking protocol. This was Andy's reasoning behind why EvStack is
designed as it is, and it sure makes sense to me.
I find it difficult to think up a console I/O operation of any
kind that cannot be handled with EvStacks, the _simplest_ of which are the
increases in available console features commonly discussed on
linux-kernel. Serial consoles, multilayer terminal emulations, font
handling, VC tweaking/redirection, code page translation, handling of
bizarre input devices, and of course any imaginable type of graphics
drawing accelerations or other features are *EASILY* handled with
practically infinite flexibility. It's all just a bunch of messages and
data, flowing from a source to a destination, with routing and tweaking
along the way.
Want to run your console output to a braille reader? No problem,
just insert a substack that routes console output to the braille-reader
driver. Want to set up your SpaceOrb 360 joystick to simulate keypresses
so it can be used with any keyboard-using game (Descent is a good example)
without the game having to know anything about the peculiarities of the
joystick? No problem, just insert the appropriate translation substack
between the joystick input driver and the console such that the console
sees keypress events as appropriate. Completely transparent, all of it,
to the kernel devices, userspace console-using apps and other stacks and
substacks.
EvStack is based upon EvPages, fixed-length chunks of data
somewhat analogous to network packets. One consequence of this is that
there need be no difference between EvStacks and their associated EvPages
in kernel and user space! it is all just a bunch of 'packets' of various
types, and as a consequence the kernel-user interface transition ceases to
be nearly the performance hit it once was. Users can insert and remove
their own user-level stacks for their own purposes without affecting
anything outside their own priviledge space, let alone having to recompile
the kernel! The kernel drivers can stay exactly the same, while what is
done with the information they provide is infintely configurable at
runtime in userspace! Another useful feature this kernel/user
transparency makes posible is very aggressive queueing and pipelineing
techniques for message handling on both the kernel and user sides. Fast,
fast, fast.
Users can insert, remove, and tweak the relationships between the
userspace stacks that interpret, massage and route the messages
originating from the kernel drivers/stacks in an unlimited number of ways.
Because of this, the kernel drivers no longer need to be nearly as
intelligent (and large, and slow, and inflexible, and potentially buggy,
and hard to maintain and document), because their communication with
userspace no longer needs to be shoehorned into any one fixed API. The
userspace stacks and libraries that will ride on top of the kernel will
take care of that end of things. The kernel input device drivers just
tell their associated handler stacks what their hardware is doing - no
frills, no interpretation, just the raw unvarnished hardware state - and
that is it. The intelligence associated with taking that device-state
info and actually doing something useful with it is elsewhere.
A great example of how this moving of device-specific intelligence
to userspace is a big win is Mesa. This is a userspace function library
that implements the OpenGL API. When the planned GGI-based Mesa port is
up and running in its final form (it already runs unaccelerated),
userspace code will link to Mesa as normal, but the Mesa library will be
EvStack aware and be able to dynamically load and unload card-specific
userspace stacks that will enable it to communicate with the kernel video
card drivers in highly flexible, optimizable and hardware-specific way.
The base kernel video drivers will blindly send and recieve messages as
described in the previous paragraph, which allows us to fine-tune the
whole communication path from the hardware -> kernel drivers -> kernel
EvStacks -> userspace EvStacks -> Mesa (and back again, as almost all
video drivers will need to send and recieve messages).
The optimal way to set this sort of communications pathway up
varies incredibly across different types of hardware, which I think leads
many GGI skeptics to doubt our ability to place all that mess under kernel
control and still have decent speed and access to card-specific features.
Doubt no more. Now, with the flexibility EvStack gives us, we can
fine-tune the stacks in question to implement this communications path
with MUCH finer-grained level of control and optimizeablility than would
ever have been possible with a traditional function-based kernel API.
The GGI project used to consider kernel graphics as the major
advance that the GGI would provide to Linux, but as the EvStack concept
has developed it has become clear to us that EvStack is the *real*
fundamental advance and that kernel graphics is just one of many
improvements to Linux made possible by EvStack. The Linux console
subsystem is aging and is due for a major overhaul. We have that
overhaul, and it certainly qualifies as "major" |->.
> > > However, when I say people are being too absolutist I mean both ways.
> > > I'm quite happy going with whomever puts out the better solution
> > > (XFree86 or GGI.) I'm just commenting on what I see.
> >
> > It hurts to be labeled an absolutist when we feel that we are
> > simply following established OS design principles.
>
> Well, there are a number of "established OS design principles" -- one
> of them is "don't do it in kernel space unless you absolutely have
> to."
Well, we have to.
> Linux has a problem with people trying to put way too many
> things in kernel space, and putting a brake on that is usually
> necessary.
This is true, but some things just plain belong behind that
kernel wall and hardware banging code is one of them. Besides, if you
read the above EvStack explanation you will see that we should be able to
greatly reduce the size of a lot of the current kernel drivers and move a
lot of their intelligence to userspace. You should be happy!
> Speaking personally; maybe you guys ought to consider making status
> summaries ("this is what we have so far") and post them.
Consider this post a status report.
> I think the
> perception is still pretty widely spread that since you didn't get
> unconditional approval from the start, you went off in a huff.
We just thought that people were getting sick of the endless
flaming and that THAT was damagin our reputation.
> That is, if you care.
We do. Dammit, we are doing what we do for the benefit of
everyone that uses Linux!
Jon