One-way Gigabit Ethernet TCP performance with Jumbo frames

From: Steffen Persvold (sp@scali.com)
Date: Thu Jan 02 2003 - 08:06:41 EST


Hi all,

Lately I've been testing out two Gigabit Ethernet adapters on Pentium 4
Xeon platforms; onboard Intel 82544GC (e1000 driver) and onboard Broadcom
BCM5701 (tg3 driver), and I'm experiencing some wierd behaviour on
one-way tests (ping-ping). The machines I'm testing is connected back to
back (i.e no switch) and are fairly fast systems (Dual Xeon 2.4 GHz, 1GB
memory) configured to use Jumbo frames (9000 bytes).

With ping-pong traffic (the attached program run with -bo, or NetPipe
default) the bandwidth performance is close to wire speed (123 MByte/sec)
and the ping-pong/2 latency is ~30us with both GbE devices.

But, when running a one-way test (where one machine only sends, and the
other only receives, i.e ping-ping) there is a serious dip in the
performance curve at ~768 bytes and the bandwidth levels out at approx
60 MByte/sec (about half of peak) regadless of application and GbE device.

However, if the benchmark applications are started at 2048 bytes (and not
0 which is default), or the MTU is set to standard 1500 bytes, there is no
such "dip" in the performance curve.

Is there a new "silly window" syndrome going on here ? Both applications
use TCP_NODELAY.

I'll appreciate any feedback, and I'm happy to assist in the debugging
process testing out patches etc.

PS

Attached you'll find a yet another "bandwidth" program which measures TCP
performance i three ways; ping-ping (one-way), ping-pong, and exchange
(two way). Should compile fine with gcc -O2, and you must start a server
process (-Ts) on one machine and a client process (-Tc -s <server>) on the
other machine.

DS

Best regards,
 --
  Steffen Persvold | Scali AS
 mailto:sp@scali.com | http://www.scali.com
Tel: (+47) 2262 8950 | Olaf Helsets vei 6
Fax: (+47) 2262 8951 | N0621 Oslo, NORWAY



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Tue Jan 07 2003 - 22:00:17 EST