Here is a copy of the thread I have been working on in
comp.protocols.tcp-ip
A fella on the list who seems to be a TCP expert, disected it, and has
comments at the end.
More Info on this is available by emailing me, unless this problem can get
fixed, I will have to pull the linux installed servers and replace with
FreeBSD 4.0, bacause I have tested FreeBSD and it worked out of the box at
about the same speed talking to the Sparc, as 2 sparcs talking to each
other, about 8-9MB/sec
Here is the dialog..
The best part is at the end... :)
Shane Brath wrote:
> I have cros-posted this here so some linux kernel people can examine
> this...
>
> The origial Is in comp.protocols.tcp
>
> Here is the deal, I have 2 servers, both news servers, both getting full
> feeds, but I am trying to get them
> to cross-feed each other. And Anytime I communicate with the linux box,
> berformance is bad, or slugish.
>
> General info: Linux is on Compaq 2500 Server, 320M/Ram, RAID disk array,
>
> 100M/TX Intel Etherpro100 PCI Card. Connected to a cisco Catalyst
> switch.
> Solaris is lates patch, Solaris 2.6 running on a Dual Processor Ultra2
> with 784M/ram, QFE FastEthernet Card, also connected to the same switch
> fabric.
>
> The network segment is swiched, but has very high traffic regardless.
>
> Ok, These machines are also conencted to the internet via multiple
> DS-3's, and When I downloaded a
> file with the solaris box from the internet size: 1.8M the solaris box
> downloaded it in 34 seconds, (53.26Kbytes/sec)
> Having the solaris box download the same file from the Linux box, I got
> the file in 2 minutes at a rate of (15.43Kbytes/sec) , this sucks, I can
>
> download from the internet faster than moving the file on same ethernet
> segment?
>
> Now, the solaris box has had it's TCP settings tuned, and cranked up,
> yet I can't find any good source for a definitive procedure for tuning
> the TCP stack on the linux box?
>
> Does anyone here have any ideas?
>
> shane brath
> shane@bratnet.net
Second Message:
Well I have tried EVERYTHING, originaly it was a ful-duplex problem,
and I fixed that with the mii-diag program.
Then I was able to get ~1.5-2/MB/sec
This was better.
But after testing 2 Solaris 2.7 Sparcs Back-to-Back on a flip, they were
able
to get 9.5-10/MB/sec on QFE cards.
Well so I thought maybee it's linux, and installed freeBSD, voila, 6.5MB/sec
- 7.9/MB/sec. With the same hardware (Test box is Compaq Despro,
PPro-180/96Mram/ tried IntelEtherExpress Pro 100 and 3com 100TX cards.) same
result :)
So this begs the question what is wrong with linux's TCP stack while talking
to a Solaris 2.6 / 2.7 box.
Well I have done a ftp From a Solaris 2.6 box via a private Flip Cable
Ethernet Network on 100 BaseTX-FD over IntelEtherExpress 100, to qfe on
solaris. got around 2/MB/sec. Fairly loaded box, newsservers, but I dumped
the tcpdump on my
ftp server in pub, and I ask that some of you kernel / tcp wizards let me
know it
this is just hardware restriction or a bug :)
ftp site: ftp://newspeer2.tds.net/pub/ratbert-dump.
Please email shane@bratnet.net for more help,
If you need instant response to a request for more info send a email to
pageshane@bratnet.net and I will check my mail :)
shane brath
Answer: From a tcp expert :)
In article <S0%P4.50$Kw4.35117@ratbert.tds.net>,
<shane@bratnet.net> wrote:
> So this begs the question what is wrong with linux's TCP stack while
talking to a Solaris 2.6 / 2.7 box.
I analyzed the dump using some tools I have developed. This analysis
is done looking at packets going from one system to another, and
calculating the timing, and delays between packets.
I can summarize it as this:
The Sun used a larger window (64K versus 32K).
The transmission used ~ 1/4th of the available bandwidth. The delays
between packets caused this poor performance. I have broken it down to
the following cases:
Case 1)
5 packets were dropped by Linux and had to be re-transmitted. This
contributed to about 30% of the elapsed time. Looking at the traces in
detail, the receiver isn't using multiple ACKS to indicate a dropped
packet. Instead, it stops ACKing new packets. The Sun has to timeout
(100+ milliseconds), and then send the missing packets. When I've
watched a Sun detect a missing packet, it does multiple ACKS of the
last good packet to indicate a packet is missing. The Linux box
doesn't do that. Delays were from 100 mseconds to 400 milliseconds in
this condition. As I said, it happend 5 times (out of 14258 packets),
but contributed to 30% of the total time..
Case 2)
The Sun had to slow down and wait for the Linux box to empty the
receive window 4451 times. This accounted to about 20% of the elapsed
time. It indicates the efficiency of the Linux TCP stack is poor when
emptying data. A faster CPU would help.
The Sun had a 320 microsecond delay between sending two packets with
"P" bits set. The "P" bit (PUSH) bit) is usually set at the end of a
buffer, so this is small, but it occured 9089 times, which contributed
to a total of 2.9 seconds. (The max was 1 millisecond). Therefore the
Sun could be faster. This is where the bottleneck is on the Sun side
it. Maybe a faster CPU would perform better. This is 30% of the time,
and 63% of the packets.
To be brief, the time is broken down into 4 cases:
20% - transmitting
30% Speed of the Sun
20% speed of the Linux box
30% Linux acknowledgement of missing packets forces a timeout
-
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@vger.rutgers.edu
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Sun May 07 2000 - 21:00:13 EST