Re: Scheduled Transfer Protocol on Linux

From: Steve Underwood (steveu@coppice.org)
Date: Sun Feb 13 2000 - 04:19:05 EST


Zachary Amsden wrote:

> [lots of stuff about the problems of scaling SMP like machines]

> That was my original point and I'm sick of going into off-topic tangents, so
> can we please stop the tirade or take it offline?

Why do you consider this to be off-topic? I think the failure of most older scaling
programs is at the heart of why the integrated file server idea is interesting.

Scaling homogenised machines higher and higher requires an increasing amount of
engineering; to develop hardware and software with smaller and smaller sales volumes;
and the performance scales less and less well the further you go. This has been the
unfortunate pattern of development over the past 30 years, and nobody seems to be
able to break out of this mould. Whether its the hardware cost of making hypercubes
to try to make an SMP like structure with low memory contention, or the software
clunkiness of endless locks, their are performance growth barriers every way you
turn.

Development wise, "traditional" styles of computer scaling just aren't going to work
for something like Linux. Too few people can afford to put more than a dual Pentium
III in front of them, so the development process won't scale if you take the
multi-processor computer path. In fact, I believe that path would lead to Linux
development being highjacked by companies with the large machines needed for
development, and that would almost certainly lead to Unix style fragmentation.

So, I believe from both the performance and project points of view, the large machine
approach is a poor choice. That makes persuing ideas like a complete file server in a
disk drive interesting. Most of us still can't develop a complete product of this
type, but we can do all the preliminary work, prior to final engineering by the drive
makers (i.e. a large part of the work will scale). It isn't true that Linux won't
embed in modest systems. I'm not sure what controller goes into most drives, but the
last time I looked the StrongARM was a serious contender. That runs Linux. Give it
some extra ROM and RAM and a LAN interface, and you might have a reasonable file
server in a disk drive. The problem I see in 2000 is that 100Mbps Ethernet is too
slow for this and GigE is too expensive. I think it will come, though. Disk drives
have gained creeping intelligence over the last 20 years. The next step is probably
the integrated file server.

Perhaps the integrated file server idea will work out, and perhaps it won't, but it
is an interesting route to explore. Repeating old SMP mistakes,and persuing a path
that locks out most current developers on cost grounds, seems doomed to failure.

Steve

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Tue Feb 15 2000 - 21:00:24 EST