Patch for serial consoles

Theodore Y. Ts'o (tytso@mit.edu)
Tue, 6 Oct 1998 00:55:07 -0400


Hi Linus,

The following patch fixes a bug in the tty layer which shows up if you
are using serial consoles and the serial console receives a hangup.
Without this patch, the refcounts at the tty layer get out of sync with
the refcounts at the serial driver, and the user gets all sorts of
warnings about tty count corruption.

The patch is not particularly clean, but it's the best that can be done
short of ripping out and rewiring a goodly amount of the tty open and
hangup processing. The problem is that I never contemplated having the
hangup routine exempt certain file descriptors based on whether they
were opened via /dev/ttyS* or via /dev/console. When the folks who
added the serial console support added this exception, this caused all
sorts of problems. This patch fixes the problem, but it is kind of
ugly.

Post 2.2, what I want to do is rewrite the tty open code so that all of
the POSIX block-til-carrier-detect code is done in the high-level tty
driver, instead of in each low-level driver. I also want to redo the
hangup processing so that it doesn't happen at hangup time, but rather
sets a flag indicating that the tty is to be hungup, and then schedule
the work to happen later, during a bottom-half handler.

But in the meantime, this patch is the simplest way of getting rid of
the bug. I'd appreciate it if you coud apply this to the tree. This
patch originally came from Miquel van Smoorenburg, and I've looked it
over carefully.

- Ted

Patch generated: on Mon Oct 5 21:19:01 EDT 1998 by tytso@rsts-11.mit.edu
against Linux version 2.1.124

===================================================================
RCS file: drivers/char/RCS/tty_io.c,v
retrieving revision 1.1
diff -u -r1.1 drivers/char/tty_io.c
--- drivers/char/tty_io.c 1998/10/06 01:18:09 1.1
+++ drivers/char/tty_io.c 1998/10/06 01:18:56
@@ -390,7 +390,9 @@
{
struct tty_struct *tty = (struct tty_struct *) data;
struct file * filp;
+ struct file * cons_filp = NULL;
struct task_struct *p;
+ int closecount = 0, n;

if (!tty)
return;
@@ -407,10 +409,13 @@
if (!filp->f_dentry->d_inode)
continue;
if (filp->f_dentry->d_inode->i_rdev == CONSOLE_DEV ||
- filp->f_dentry->d_inode->i_rdev == SYSCONS_DEV)
+ filp->f_dentry->d_inode->i_rdev == SYSCONS_DEV) {
+ cons_filp = filp;
continue;
+ }
if (filp->f_op != &tty_fops)
continue;
+ closecount++;
tty_fasync(-1, filp, 0);
filp->f_op = &hung_up_tty_fops;
}
@@ -470,7 +475,17 @@
tty->session = 0;
tty->pgrp = -1;
tty->ctrl_status = 0;
- if (tty->driver.hangup)
+ /*
+ * If one of the devices matches a console pointer, we
+ * cannot just call hangup() because that will cause
+ * tty->count and state->count to go out of sync.
+ * So we just call close() the right number of times.
+ */
+ if (cons_filp) {
+ if (tty->driver.close)
+ for (n = 0; n < closecount; n++)
+ tty->driver.close(tty, cons_filp);
+ } else if (tty->driver.hangup)
(tty->driver.hangup)(tty);
unlock_kernel();
}
@@ -1243,6 +1258,7 @@
if (!c)
return -ENODEV;
device = c->device(c);
+ filp->f_flags |= O_NONBLOCK; /* Don't let /dev/console block */
noctty = 1;
}
#ifdef CONFIG_UNIX98_PTYS

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/