strange loopback tcp performance problem

Andrew Tridgell (tridge@samba.anu.edu.au)
Fri, 23 Oct 1998 14:47:30 +1000


While running some speed tests on Samba I came across a severe
performance problem with the current 2.1 TCP stack on loopback.

The problem is that the tcp throughput varies between about 200 k/sec
and 25 MB/sec on my unlodaded K6-200 running 2.1.125. A 2.0.34 kernel
delivers a consistent 16 MB/sec on a similar machine.

I've written a small test program which is available at
ftp://samba.anu.edu.au/pub/tridge/misc/socklib.tgz

To run the test do the following:

./sock_source -b 1024 &
./sock_sink -H localhost

this will give you a printout of the speed as seen by the server and
client every 2 seconds (they should be roughly the same). Wait for a
large number of lines to be printed then ^C and restart sock_sink (but
not sock_source). Again wait for a bunch of output then restart it
again. Observe the huge variance in throughput.

If you want to you can play with socket options and buffer sizes on
the command line using -t an -b. For example:

./sock_sink -t "TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=16384 IPTOS_THROUGHPUT"

The critical thing appears to be that the sender is doing 1k writes
(the -b 1024 option above). I expected this to make a small
difference, but not a factor of 100. And why the huge varience between
runs?

A sample of what I see follows, although I've seen much greater
variation that this run (the maximum variation I've seen is 200 k/sec
-> 25 MB/sec)

bash$ ./sock_source -b 1024 &
[1] 14798
bash$ port=7001 options=[]
waiting for connection
bash$ ./sock_sink -H localhost
host=localhost port=7001 options=[]
accepted
waiting for connection
0.793292 MB/sec
0.79679 MB/sec
0.687321 MB/sec
0.715289 MB/sec
0.756733 MB/sec
0.740292 MB/sec
0.743484 MB/sec
0.728355 MB/sec
0.747519 MB/sec
0.740157 MB/sec
0.740038 MB/sec
0.733245 MB/sec
0.934274 MB/sec

bash$ ./sock_sink -H localhost
host=localhost port=7001 options=[]
accepted
waiting for connection
20.9057 MB/sec
20.3226 MB/sec
20.9555 MB/sec
20.9682 MB/sec
1.13213 MB/sec
1.14311 MB/sec
0.722275 MB/sec
0.800895 MB/sec
0.736669 MB/sec
0.682532 MB/sec
0.740097 MB/sec
0.74058 MB/sec
0.744444 MB/sec
0.743216 MB/sec

0.828012 MB/sec
bash$ ./sock_sink -H localhost
host=localhost port=7001 options=[]
accepted
waiting for connection
5.69272 MB/sec
5.66049 MB/sec
3.74547 MB/sec
3.74673 MB/sec
0.730152 MB/sec
0.729695 MB/sec
0.722755 MB/sec
0.739306 MB/sec

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/