Look Larry, I consider you a good person; but please note that repeating
things like "you are a beginner", or "I've been there before you", or
"you don't specifically say what you mean" (my favourite among your comments)
will not help to perform a good exchange of ideas at all. I've been trying
to be tolerating till now, but as you know very well, I don't have to.
I have tried to explain my position repeatedly, and this is starting to
become frustrating.
lm@bitmover.com (Larry McVoy) wrote:
>OK, so let me summarize.
Please read other people's comments more carefully before making such
judgements on them.
> . You have no experience with cluster based systems.
My DIPC has been enabling people to build clusters for the last two years.
How much practical work have _you_ done for Linux? BTW, I humbly suggest you
read DIPC's documents. You may find answers to some the problems you are
thinking about now.
> . You have no experience as a systems architect.
See above. Are you one of those persons who can predict on people by reading
a few of their emails? I hope your approach to scientific matters is
different.
> . You state that you aren't interested on past experiences in this area.
In my last email to you I clearly said that others might have had different
goals to reach when designing their systems. I am not a follower when I don't
believe the move is in the right direction.
Actually, I believe I am using past experience more than you are. I do this
by observing how the job of current generation of programmers has been made
simpler by hidding some details about the operation of the system from them.
> . You aren't able to do any work yourself.
Nice choice of words by you! I am able to do almost anything I decide is
worth my efforts. Currently I don't have the time to get involved in a new
project.
>And yet you find yourself qualified to tell other people how the job ought
>to be done. Don't you find that the slightest bit presumptuousness?
Of course I am qualified (much more than some other people I've come to
know on this list). Do you want me to present my ideas with C source code?
Sorry if my opinions don't match yours, but that is not a criteria for
"disqualification"
>I personally hate it when grey haired architects tell me my ideas are stupid
>but don't tell me the right answer. So here's a little sketch of what I think
>is the right answer for clustering. I'm working this up on my web site, you
>might want to check that in a week or two.
I hope by "grey haired architects" you didn't mean me. I never said your
ideas are stupid, but I say there are some better ways of doing things. I
_have_ explained my ideas time and again. The problem in this discussion
seems to be that you have some model of how things should be done, and try
to fit my proposals in that model. Well, they don't fit, but their basics
should be clear by now. I'll explain _again_:
*) The best way to add new functionality is by re-using and modifying the
mechanisms with which programmers are already familiar. This will enable more
people to use the new facilities, without requiring more training time.
*) Continuing the above argument, the new services should better come from
the same places as the services they are extending. If the kernel used to
handle the creation of new processes in a single machine (via the fork), it
it better if the kernel does so in a distributed environment too (again, by
fork). The program _can_ have influence by giving hints (though ideally that
should not be necessary). If another service we want to extend comes from a
user-space library, then we should modify this user-space library.
It is a fact that process management is handled by the kernel, with very
little attention from the part of the application itself. If you object to
distributed process mamanegment (including, but not limited to process
migration) being done by the kernel, then why don't you object to pre-emptive
process scheduling being done by the kernel? Doesn't the application "know
better" about "when" and "for how long" it should run? Others before you have
hidden these issues, so you can program a bit easier. Why don't you use some
"past experience"??
*) Programs should employ the new services with minimal change in the source
code. They should not care about things like process migration. I wonder
how you would like it if an operation like simply opening a file required
the involvemnet of the programmer in some kernel-level cache managemnet
decision makings. The programmer should not care about such things: One
should learn a bit from past experience.
*) All the programs should be able to use these mechanisms. A Scientific
application will probably benefit most, but so will multi-player games. Let
anybody who decides to use the new services do it with ease. The actual
decision of using them is not up to you and me, but to the programmer, and
more programmers will use them if they are easy to use: As a general rule
(applicable here), the less you have to do to accomplish something, the
easier it will be to do. As another general rule (also applicable here)
the easier using something is, the more people will start using it.
>Things which are bad:
> . Distributed shared memory. It has no failure model [...]
Replication.
> . Remote procedure calls. RPCs block. [...]
RPC is useful for increasing the transparency. Considering that latency will
probably be the most important factor in performance, I wonder how you say
RPC is any slower. You have to setup a network connection _no_matter_ the
programming model and mechanism. RPC will at most add a bit of computational
penalty for handling arguments, but RPC is much eaiser to use for the
programmer, as they preserve most of the syntax and semantics of ordinary
procedure calls.
> . Process migration. Sounds good, but is way too costly to be of any
> use. [...]
Disagree. I am so happy many OS designers did not listen to those (like
Mac-OS people) who said pre-emptive process scheduling is just an overhead
that only makes the OS more complicated without being of any use to most
programs.
Process migration is the best way for efficient use of a cluster's
resources. Isn't this efficient resource-usage one of the main goals of
employing a cluster? One of the other goals is fault tolerance. Again,
process migration can help.
> . Dynamic load balancing across machines.
It is necessary for process migration, and as some of us think process
migration is very useful, some of us say we should have it.
> . Remote fork(). It's a nice idea, but it doesn't work well in practice.
No. A lot depends on the way it is implemented, and also on what semantics
are going to be preserved when going from a normal fork to a remote fork.
> . Security.
Interesting. You are so concernced about fault-tolerance and yet you think
security is a bad thing? talk about experience!! For most people in today's
networked computing environments security comes first. They might tolerate
occasional crashes, but not at all a break-in. Of course unless we want to
turn back the clock and ask everybody to put their clusters in a safe with
no network connection to the rest of the world.
Most organizations (and many individuals) already have a lot of networked
PCs or workstations available. The goal should be to enable them to use the
combined power of their machines. They will not easily give up the security
of their computers and the data on them.
>Things which are good:
> . Local and global names.
> - processes
Agreed.
> - /dev vs /gdev
> /dev/whatever means a device on this node.
> /gdev/whatever may or may not be on this node. Note that
> /gdev has entries /only/ for real devices, unlike /dev.
Why for God's sake do you want to separate these two? The kernel developer
will have to handle the different cases of a local device and a remote device
anyway, but I say don't require the programmer to know about the difference.
Let the programs deal with only one form. _If_ they really need only local or
only remore service, they can give a hint. The rest should not be different.
> - /tmp vs /gtmp
> - /proc vs /gproc
The same as above.
> . Messages and message queues.
This is different from normal UNIX programming. An example is when you
want to use System V messages. If the message buffer inside the kernel is
full, the process wanting to deliever another message is blocked. Why do you
think people with more experiece than you were not very happy about
non-blocking operation?
> Messages also work well in a cluster because they give you positive
> notification that the work is done. If you try and use distributed
> shared memory, you end up using messages anyway. Consider a producer
> consumer problem in DSM. The producer puts the data in memory.
> How does the consumer know that the data is ready? Well, it has to
> get notified. What is that? That's a message. How do you do a
> message in DSM?
Synchronization problems have been investigated by "grey haired architects"
before you and I were born. Read some OS books.
You could also refer to DIPC's documents to see how such problems are solved
in a "practical system"
> . Remote exec.
> . Load balancing.
> . Cache coherent filesystems.
Agreed.
> . System area networks (security). Rather than layer secure protocols on
> top of the network, you just use what the industry calls a system area
> network (SAN). A SAN is a private network with no gateways. Traffic
> on the SAN is trusted, just like traffic on a SMP backplane is trusted.
This will not cover all needs, please see my comments about security.
You don't agree with my ideas? That is really OK with me. I won't label you
with anything negative. I just consider you a respectful person with
different goals and priorities in mind. Acting the other way won't make
anybody's ideas any better.
I wish you good luck with your efforts on Linux.
-Kamran Karimi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu