Linux tcp/ip code has trouble with async network I/O notification,

Wim Ten Have (wtenhave@sybase.com)
Thu, 3 Sep 1998 00:24:08 +0200 (MET DST)


Hi,

Perhaps I should directly mail you. I feel truely sorry if this
message is directed to the wrong group of people. In case it is
please ignore and delete.

I think that there is a major ;-) problem around in the Linux TCP/IP
code when using Async (SIGIO) controlled network code setups.

The re pro code below my signature may proof this. Forgive me my
bad and dirty coding it was only mend to show/expose/proof the
problem. Looking at the kernel code affected I think it will come
down to the net/ipv4/tcp_input.c area where use tcp_rcv() ->
sock_wake_async() to notify before the conditions on the
tcp_select() are clear.

The idea of the re pro is that a certain server process needs ASYNC
I/O notification when recv network I/O is possible. Under Linux
this only works "well" for local clients though clients routed via
external physical devices have difficulties.

Read and try to understand the provided code below.

Start the program with
$ ./srv &

And perform a telnet to the hard coded port
$ telnet `uname -n` 6543

When it worked you'll see
Received descr = 1[0] SUCCEEDED

Start the program again with
$ ./srv &

And perform a telnet to IP and hard coded port from an other machine

The server process will report with
Received descr = 0[0] FAILED

Needless to say that the program works as expected under BSD and
Solaris and other UNIX (socket) platforms where I like it to get it
to work as expected under Linux.

I've verified the problem to be around in all kernels.
I.e. linux-2.0.35 up to linux-2.1.117

Changing the select to block or with a time-out value is *not* an
option in my coding model. And the select() should be able to find
the file-desc after the NETIO notification (SIGIO).

Some understanding wrt this problem I currently hold is that the
SIGIO is posted before the socket connection state is set to
ESTABLISHED making the select(2) miss the available data. Local
clients have the advantage that their data is already available so I
think they are served by accident.

:-( :-( :-(

Please help,
-- Wim ten Have.

/*
** Repro to proof that there is some problem in the Linux kernels
** with SIGIO driven netio notification. When operating on none
** localhost network setups. Ie remote connections fail to notify
** in correct order.
**
** Author: Wim ten Have. <wtenhave@sybase.com>
** Date: Mon Aug 24 18:14:26 MET DST 1998
**
** Disclaimer:
** This code is dirty and programmed in 20 minutes.
**
** Usage:
** start the server program and use telnet from an other window
** to the programmed port 6543
**
** $ cc -o linux_srv linux_src.c
** $ ./linux_srv &
** $ telnet `uname -n` 6543
** Received descr = 1[0] SUCCEEDED
**
** Do the same from an other machine, ie make the connection work
** via the ethernet interface.
**
** $ ./linux_srv &
** $ telnet <ip-of-your-server-machine> 6543
** Received descr = 0[0] FAILED
*/
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
/* if running under solaris */
#if !defined(linux)
#include <sys/sockio.h>
#endif
#include <netinet/in.h>
#include <arpa/inet.h>

#include <signal.h>
#include <fcntl.h>
#include <sys/file.h>
#include <sys/ioctl.h>
#include <sys/time.h>

extern int errno;
void sig();

int sd, nsd, pid;
int testing = 1;

int service ()
{
int n;
fd_set rset;
struct timeval *timo;
struct timeval timeout;
timo = &timeout;
timo->tv_sec = 0;
timo->tv_usec = 0;

FD_ZERO (&rset);
FD_SET (sd, &rset);

if ((n = select(sd+1, &rset, NULL, NULL, timo)) < 0)
perror("select"), exit(1);

printf ("Received descr = %d[%d] %s\n",
n, errno, n ? "SUCCEEDED" : "FAILED");

testing = 0;
}

main(ac, app)
int ac;
char **app;
{
int pgrp;
int opt = 1;
struct sockaddr_in saddr;

if (signal (SIGIO, service))
perror("signal"), exit(1);

if ((sd=socket(AF_INET, SOCK_STREAM, 0)) == -1)
perror("socket"), exit(1);

bzero((char *)&saddr, sizeof(saddr));
saddr.sin_family = AF_INET;
saddr.sin_addr.s_addr = htonl(INADDR_ANY);
saddr.sin_port = htons(6543);

pgrp = getpid();
if (ioctl (sd, SIOCSPGRP, (char *)&(pgrp)) < 0)
perror("ioctl"), exit(1);
if (fcntl (sd, F_SETOWN, pgrp) < 0)
perror("F_SETOWN"), exit(1);
if (fcntl (sd, F_SETFL, FNDELAY|FASYNC) < 0)
perror("fcntl"), exit(1);
if ((setsockopt(sd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)) < 0)
|| (setsockopt(sd, SOL_SOCKET, SO_KEEPALIVE, &opt, sizeof(opt)) < 0))
perror("setsockopt"), exit(1);

if (bind(sd, (struct sockaddr_in *)&saddr, sizeof(saddr)) == -1)
perror("bind"), exit(1);

listen(sd, 5);

while (testing) {
sleep (1);
}
close (nsd);
close (sd);
exit (errno);
}

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.altern.org/andrebalsa/doc/lkml-faq.html