multicast BROKEN in 2.1!

JSULMONT.US.ORACLE.COM (JSULMONT@us.oracle.com)
29 Jan 98 13:08:20 -0800


--=_ORCL_17450038_0_0
Content-Transfer-Encoding:7bit
Content-Type:text/plain; charset="us-ascii"


Hi.

The MULTICAST seems to be broken in 2.1.X with X >= 79
The problem happens when several processes ON A SAME MACHINE
join a multicast address and pretend receiving messages SENT by
another on process the SAME machine to the IP-multicast address/PORT.
Since I do not have 2 Linux boxes here, I used a SPARC running
solaris 5.5.1 as "remote".

The code of the two programs used for the test is adapted from the
code published with Stevens book V2. I've hacked it sorry if its
ugly. The point is that the receiver are never going to try to send
and vice versa. The sender does not joins. The interface choice is left
up to the kernel. You will find it attached.

The test consists in the following script:
bash# exec tcsh
tcsh# ./recv 226.10.10.10 1234 >& R1
tcsh# ./recv 226.10.10.10 1234 >& R2
tcsh# ./recv 226.10.10.10 1234 >& R3
tcsh# ./recv 226.10.10.10 1234 >& R4
tcsh# ./recv 226.10.10.10 1234 >& R5
tcsh# ./recv 226.10.10.10 1234 &


2.0.32: single device configured: eth0 (not as module but in kernel:
eth0: 3Com 3c905 Boomerang 100baseTx at 0xecc0, 00:c0:4f:a3:04:ef, IRQ 14
eth0: MII transceiver found at address 24.
eth0 is running MULTICAST and the 224.0.0.0 route is on eth0
(although this string was cut & past from dmesg from a 2.1.79 kernel).
then it works OK:
tcsh# head -2 R1 R2 R3 R4 R5
==> R1 <==
from 138.2.141.124.1030: bloom.us.oracle.com, 291 (0)
from 138.2.141.124.1030: bloom.us.oracle.com, 291 (1)
==> R2 <==
from 138.2.141.124.1030: bloom.us.oracle.com, 291 (0)
from 138.2.141.124.1030: bloom.us.oracle.com, 291 (1)
==> R3 <==
from 138.2.141.124.1030: bloom.us.oracle.com, 291 (0)
from 138.2.141.124.1030: bloom.us.oracle.com, 291 (1)
==> R4 <==
from 138.2.141.124.1030: bloom.us.oracle.com, 291 (0)
from 138.2.141.124.1030: bloom.us.oracle.com, 291 (1)
==> R5 <==
from 138.2.141.124.1030: bloom.us.oracle.com, 291 (0)
from 138.2.141.124.1030: bloom.us.oracle.com, 291 (1)
This is OK, I can leave it for hours, I do not miss a single message
that was sent.

I have replayed this test with the dummy driver: OKAY.
The kernel has 3 interfaces: lo0 dummy eth0, the route 224.0.0.0 is
changed onto dummy. Went fine.

2.1.79: my system has 3 devices: eth0, fddi0 lo0.
fddi0: DEFPA at I/O addr = 0xEC00, IRQ = 11, Hardware
addr=00-00-F8-BD-CC-61
eth0: 3Com 3c905 Boomerang 100baseTx at 0xecc0, 00:c0:4f:a3:04:ef, IRQ 14
8K word-wide RAM 3:5 Rx:Tx split, autoselect/MII interface.
eth0: MII transceiver found at address 24.
Enabling bus-master transmits and whole-frame receives.
the routes are as following:
138.2.140.0 0.0.0.0 255.255.255.0 U 0 0 0
fddi0
138.2.141.0 0.0.0.0 255.255.255.0 U 0 0 0
eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
224.0.0.0 0.0.0.0 224.0.0.0 U 0 0 0
eth0
0.0.0.0 138.2.140.1 0.0.0.0 UG 0 0 0
fddi0

then the same test yields:
tcsh# head -2 R1 R2 R3 R4 R5
==> R1 <==
from 138.2.141.124.1024: bloom.us.oracle.com, 2955 (14)
from 138.2.141.124.1024: bloom.us.oracle.com, 2955 (68)

==> R2 <==
from 138.2.141.124.1024: bloom.us.oracle.com, 2955 (14)
from 138.2.141.124.1024: bloom.us.oracle.com, 2955 (68)

==> R3 <==
from 138.2.141.124.1024: bloom.us.oracle.com, 2955 (14)
from 138.2.141.124.1024: bloom.us.oracle.com, 2955 (68)

==> R4 <==
from 138.2.141.124.1024: bloom.us.oracle.com, 2955 (14)
from 138.2.141.124.1024: bloom.us.oracle.com, 2955 (68)

==> R5 <==
from 138.2.141.124.1024: bloom.us.oracle.com, 2955 (14)
from 138.2.141.124.1024: bloom.us.oracle.com, 2955 (68)
When a datagram is delivered to a process then it is delivered
onto ALL the datagrams that have join the multicast group.
But the receive of datagrams seems to be random.
When I run the ./send program on a SPARC, then everything
works fine: they all receive without missing a dgrm.
Alike, broken with the dummy.


2.1.82: same as above but running with only the FDDI interface
hard configured in the kernel.
This is exactly the same thus the frequency at which
messages are received when they are is way higher (one
every 2 or 3 sent).
When run from the remote SUN everything works fine.



My understanding is that the driver is not relevant to the problem
(dummy.c, 3c59x.c, defxx.c) and the number of interface is not relevant.
Could somebody tell me which code to look at ... or fix it :-)
I do not have any knowledge of the 2.1 tree past 79... I haven't
been able to find a bug report on the matter as well. This is too
GROSS that I keep believing that I've screwed something. But cannot
find what. Somebody help, please (otherwise I'll have to resume
my developments on the UGLY solaris box)

Thanks for your help.

PS. I'm running a DELL Workstation 400 with 128Mb and 2X300Mhz PentiumII
SCSI disks, and a CDROM IDE.
and 2.1.81. The SPARC is **TOASTED**.




------------------------------------------------------------------------
Jean-Marie SULMONT 1000 SW Broadway
Software Engineer Suite 1200
Oracle Corp Portland, OR 97205
jsulmont@us.oracle. (503) 525-8057
------------------------------------------------------------------------

--=_ORCL_17450038_0_0
Content-Transfer-Encoding:7bit
Content-Type:text/plain;
name="send.c";
charset="us-ascii"
Content-Disposition:attachment;
filename="send.c"

#include <stdio.h>
#include <errno.h>
#include <assert.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/utsname.h>
#include <netdb.h>
#include <netinet/in.h>
#include <net/if.h>
#include <linux/sockios.h>

#define SENDRATE 2
#define MAXLINE 1024

void
send_all (int sendfd, struct sockaddr *sadest, socklen_t salen)
{
static char line[MAXLINE]; /* hostname and process ID */
struct utsname myname;
int c;
static int COUNT = 0;

if (uname (&myname) < 0)
{
perror ("uname error");
exit (1);
}
for (;;)
{
c = snprintf (line, sizeof (line), "%s, %d (%d)\n",
myname.nodename, getpid (), COUNT++);
if (sendto (sendfd, line, c, 0, sadest, salen) != c)
{
perror ("sendto error");
exit (1);
}
sleep (SENDRATE);
}
}

int
udp_client (const char *host, const char *serv, void **saptr, socklen_t * lenp)
{
int sockfd, n;
struct addrinfo hints, *res, *ressave;

bzero (&hints, sizeof (struct addrinfo));
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_DGRAM;

if ((n = getaddrinfo (host, serv, &hints, &res)) != 0)
{
fprintf (stderr, "udp_client error for %s, %s\n", host, serv);
exit (1);
}
ressave = res;

do
{
sockfd = socket (res->ai_family, res->ai_socktype, res->ai_protocol);
if (sockfd >= 0)
break;
}
while ((res = res->ai_next) != NULL);

if (res == NULL)
{
fprintf (stderr, "udp_client error for %s, %s", host, serv);
exit (1);
}
*saptr = (struct sockaddr *) malloc (res->ai_addrlen);
assert (*saptr != NULL);
memcpy (*saptr, res->ai_addr, res->ai_addrlen);
*lenp = res->ai_addrlen;
freeaddrinfo (ressave);
return (sockfd);
}

int
main (int argc, char **argv)
{
int sendfd, recvfd;
int on = 1;
socklen_t salen;
struct sockaddr *sasend, *sarecv;

if (argc != 3)
{
fprintf (stderr, "usage: %s IP-multicast-address port#\n", argv[0]);
exit (1);
}
sendfd = udp_client (argv[1], argv[2], (void **) &sasend, &salen);
assert (sendfd >= 0);
send_all (sendfd, sasend, salen);
}

--=_ORCL_17450038_0_0
Content-Transfer-Encoding:7bit
Content-Type:text/plain;
name="recv.c";
charset="us-ascii"
Content-Disposition:attachment;
filename="recv.c"

#include <stdio.h>
#include <errno.h>
#include <assert.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/utsname.h>
#include <netdb.h>
#include <netinet/in.h>
#include <net/if.h>
#include <linux/sockios.h>

#define SENDRATE 2
#define MAXLINE 1024

char *
sock_ntop (const struct sockaddr *sa, socklen_t salen)
{
char portstr[7];
static char str[128];

struct sockaddr_in *sin = (struct sockaddr_in *) sa;

if (inet_ntop (AF_INET, &sin->sin_addr, str, sizeof (str)) == (char *) 0)
return (NULL);
if (ntohs (sin->sin_port) != 0)
{
snprintf (portstr, sizeof (portstr), ".%d", ntohs (sin->sin_port));
strcat (str, portstr);
}
return (str);
}

void
recv_all (int recvfd, socklen_t salen)
{
int n;
char line[MAXLINE + 1];
socklen_t len;
struct sockaddr *safrom;

safrom = (struct sockaddr *) malloc (salen);
assert (safrom != NULL);
for (;;)
{
len = salen;
n = recvfrom (recvfd, line, MAXLINE, 0, safrom, &len);
if (-1 == n)
{
perror ("recvfrom error");
exit (1);
}
line[n] = 0; /* null terminate */
fprintf (stderr, "from %s: %s", sock_ntop (safrom, len), line);
}
}

int
mcast_join (int sockfd, const struct sockaddr *sa,
socklen_t salen, const char *ifname, u_int ifindex)
{
struct ip_mreq mreq;
struct ifreq ifreq;

memcpy (&mreq.imr_multiaddr,
&((struct sockaddr_in *) sa)->sin_addr,
sizeof (struct in_addr));

if (ifindex > 0)
goto doioctl;
else if (ifname != NULL)
{
strncpy (ifreq.ifr_name, ifname, IFNAMSIZ);
doioctl:
if (ioctl (sockfd, SIOCGIFADDR, &ifreq) < 0)
return (-1);
memcpy (&mreq.imr_interface,
&((struct sockaddr_in *) &ifreq.ifr_addr)->sin_addr,
sizeof (struct in_addr));
}
else
mreq.imr_interface.s_addr = htonl (INADDR_ANY);

return (setsockopt (sockfd, IPPROTO_IP, IP_ADD_MEMBERSHIP,
&mreq, sizeof (mreq)));
}

int
udp_client (const char *host, const char *serv, void **saptr, socklen_t * lenp)
{
int sockfd, n;
struct addrinfo hints, *res, *ressave;

bzero (&hints, sizeof (struct addrinfo));
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_DGRAM;

if ((n = getaddrinfo (host, serv, &hints, &res)) != 0)
{
fprintf (stderr, "udp_client error for %s, %s\n", host, serv);
exit (1);
}
ressave = res;

do
{
sockfd = socket (res->ai_family, res->ai_socktype, res->ai_protocol);
if (sockfd >= 0)
break; /* success */
}
while ((res = res->ai_next) != NULL);

if (res == NULL) /* errno set from final socket() */
{
fprintf (stderr, "udp_client error for %s, %s", host, serv);
exit (1);
}

*saptr = (struct sockaddr *) malloc (res->ai_addrlen);
assert (*saptr != NULL);
memcpy (*saptr, res->ai_addr, res->ai_addrlen);
*lenp = res->ai_addrlen;
freeaddrinfo (ressave);
return (sockfd);
}

int
main (int argc, char **argv)
{
int sendfd, recvfd;
int rc, on = 1;
socklen_t salen;
struct sockaddr *sasend, *sarecv;

if (argc != 3)
{
fprintf (stderr, "usage: %s IP-multicast-address port#\n", argv[0]);
exit (1);
}

sendfd = udp_client (argv[1], argv[2], (void **) &sasend, &salen);
assert (sendfd >= 0);

recvfd = socket (sasend->sa_family, SOCK_DGRAM, 0);
assert (recvfd >= 0);
assert (setsockopt (recvfd, SOL_SOCKET, SO_REUSEADDR, &on, sizeof (on))
!= -1);
sarecv = (struct sockaddr *) malloc (salen);
assert (sarecv != NULL);
memcpy (sarecv, sasend, salen);
if (-1 == bind (recvfd, sarecv, salen))
{
perror ("bind failed\n");
exit (1);
}
rc = mcast_join (recvfd, sasend, salen, NULL, 0);
assert(rc != -1);
recv_all (recvfd, salen);
}

--=_ORCL_17450038_0_0--