vhangup() and PTY

Henrik Nordstrom (henrik.nordstrom@ida.his.se)
Thu, 29 Feb 1996 18:47:59 +0100


This is a multipart MIME message.

--===_0_Thu_Feb_29_18:08:43_MET_1996
Content-Type: text/plain; charset=us-ascii

It seems like there is a race condition in vhangup() on a PTY. Sometimes
it can fail to stop other processes from having the pty open.

System: 386dx/33 8Mb, Linux1.3.58.

I guess that the following (or similar) is happening:
1. One or more processes try to open the slave side, and are suspended
inside the open() call (*).
2. The master side is opened, and the processes in 1 are notified.
3. The slave side is opened and vhangup() issued. vhangup closes
all open references to the pty and kills -HUP all processes with the
pty as controlling terminal.
4. A process from 1 wakes up and continues the open.

(*) I am not sure that the processes are really suspended. They all take
as much CPU as they can, in system time only.

If I insert a sleep somewhere between the master side open and the
vhangup() I haven't been able to reproduce it.

telnet and rlogin/shell does quite a lot between the open of the master
side and the vhangup() (done by login) and it is not likely that this
affect them except possibly under very heavy load.

My apologies if this is fixed in a later release of the kernel.

For simplicity here is a small program that opens a pty, issues vhangup()
andstarts a shell on the pty. This real program also sets the permissions
on the pty to root 0600 before the fork, and sets up the line mode
of the slave side before executing the program, changes user id and
so on.. but the same problem exists in the bigger version.

---
Henrik Nordstrom
University of Skoevde
Sweden

--===_0_Thu_Feb_29_18:08:43_MET_1996 Content-Type: text/plain; charset=us-ascii Content-Description: ptytest.c

#include <sys/time.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <signal.h> #include <unistd.h>

#include <termios.h>

int main(int argc,char **argv) { int m,s,c; fd_set fds; char master[]="/dev/ptyq0"; char slave[]="/dev/ttyq0"; m=open(master,O_RDWR); if(fork()==0) { close(m); ioctl(0,TIOCNOTTY,0); setsid(); s=open(slave,O_RDWR); signal(SIGHUP,SIG_IGN); vhangup(); signal(SIGHUP,SIG_DFL); s=open(slave,O_RDWR); /* start ithe child... */ dup2(s,0); dup2(s,1); dup2(s,2); execl("/bin/sh","sh",NULL); } sleep(1); /* Wait for slave side end to start */ /* Forward data to and from the pty */ while(1) { FD_ZERO(&fds);FD_SET(0,&fds);FD_SET(m,&fds); select(FD_SETSIZE,&fds,NULL,NULL,NULL); if(FD_ISSET(0,&fds)) { if(read(0,&c,1)!=1) goto done; if(write(m,&c,1)!=1) goto done; } if(FD_ISSET(m,&fds)) { if(read(m,&c,1)!=1) goto done; if(write(1,&c,1)!=1) goto done; } } done: exit(); }

--===_0_Thu_Feb_29_18:08:43_MET_1996

--===_0_Thu_Feb_29_18:08:43_MET_1996--