Hello.
We have been measuring performances of SpecWeb96 benchmark program.
The server machine had a 500MHz Pentium III (Katmai) and the client
had a 600MHz Pentium III (Coppermine). They were connected directly
via 100Base-T (ipv4) with a reverse cable.
When we used an APACHE (1.3.9) HTTP server that used parallel acceptance
policy (See Appendix), we obtained the following strange results.
Linux of Linux of (Benchmark results)
Server Client Throughput Response
Machine Machine [HTTPops/s] [ms]
----------------------------------------
2.2.14 - 2.2.14 312 25.5
2.2.14 - 2.3.49 567 14.1
2.3.41 - 2.2.14 614 13.0
2.3.41 - 2.3.49 606 13.2
That is, the benchmark result was extremely bad only when the
server and the client used Linux 2.2.14.
We investigated the cause of this strange results and found that the
__tcp_v4_lookup() function in linux/net/ipv4/tcp_ipv4.c had a problem:
When the server receives a TCP/IP packet whose SYN bit is 1, which
means a request of TCP/IP connection, the __tcp_v4_lookup() sometimes
find a TIME_WAIT'er socket whose sock_readers is not zero.
If such socket is used for the TCP/IP-connection request, the server
does not reply to the request, because the tcp_timewait_state_process()
routine (linux/net/ipv4/tcp_input.c) does not call
'sk->tp_pinfo.af_tcp.af_specific->conn_request()' when 'sk- >sock_readers'
is not zero. This results in the timeout on the connection request and the
client will re-send a connection request.
This phenomena seems to occur when the utilization of the CPU is high,
close to 100%.
Since Linux 2.3.49 uses TCP/IP port number up to 65535, the time
length from using a socket to (re)using a socket that has the same port
number to that is longer than that of Linux 2.2.14, which uses TCP/IP
port number up to 32767. This is the reason why SpecWeb96 performance
was good when the client used Linux 2.3.49. (TIME_WAIT'er sockets were
gone.)
We then made and applied the following patch to Linux 2.2.14 and measure
SpecWeb96 performance.
--------
diff -urN tcp_ipv4.c.org tcp_ipv4.c
--- tcp_ipv4.c.org Thu Feb 17 16:22:52 2000
+++ tcp_ipv4.c Fri Mar 31 22:15:13 2000
@@ -432,7 +432,8 @@
}
/* Must check for a TIME_WAIT'er before going to listener hash. */
for(sk = tcp_ehash[hash+(tcp_ehash_size/2)]; sk; sk = sk->next)
- if(TCP_IPV4_MATCH(sk, acookie, saddr, daddr, ports, dif))
+ if(TCP_IPV4_MATCH(sk, acookie, saddr, daddr, ports, dif) &&
+ atomic_read(&sk->sock_readers) == 0 )
goto hit;
sk = tcp_v4_lookup_listener(daddr, hnum, dif);
hit:
--------
The result was:
Linx of Linux of (Benchmark results)
Server Client Throughput Response
Machine Machine [HTTPops/s] [ms]
----------------------------------------
2.2.14 - 2.2.14 625 12.8
Thus, by applying this patch, an APACHE HTTP server and Linux 2.2.14
running on a server that had one CPU can outperform Linux 2.3.14 and
the APACHE running on the same machine.
I'm afraid that this patch results in the violation of the
specification of TCP/IP regarding the handling of TIME_WAIT'er socket.
Unfortunately, I'm not sure about that.
----------------------------------------------------------------
Appendix.
To build an APACHE server using parallel acceptance policy,
replace the following statements in apache_1.3.9/src/include/ap_config.h
--------
#ifndef USE_FLOCK_SERIALIZED_ACCEPT
#define USE_FCNTL_SERIALIZED_ACCEPT
#endif
--------
by '#define NO_SERIALIZED_ACCEPT', and make.
By this modification, more than one HTTPd processes will make
accept() system calls at the same time.
--------
name: Takashi Richard Horikawa
(NEC C&C Research Lab.s)
E-mail: horikawa@ccm.cl.nec.co.jp
Tel.: +81-44-856-2079
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Fri Mar 31 2000 - 21:00:29 EST