Re: Problems in tty_ioctl?

tytso@mit.edu
Wed, 31 Dec 1997 18:03:53 -0500


Date: Mon, 29 Dec 1997 20:04:09 +0200 (EET)
From: T Taneli Vahakangas <vahakang@cs.Helsinki.FI>

I am using kernel version 2.1.76. Today, I tried a program called
9term (it's a terminal emulator for X) and wondered why did it exit
right after pressing a key. To my amusement, the program died because
of an oops (attached, but looks weird). The kernel stumbled in
tty_ioctl().

The following patch cures the problem, but probably not the cause:

The patch is almost right, actually. The main problem is that you
should never return ENOIOCTLCMD to the user-level code. Enclosed is a
better patch, which should work against 2.1.76 even though it was made
against 2.1.75.

- Ted

Patch generated: on Wed Dec 31 17:54:32 EST 1997 by tytso@rsts-11
against Linux version 2.1.75

===================================================================
RCS file: drivers/char/RCS/tty_io.c,v
retrieving revision 1.1
diff -u -r1.1 drivers/char/tty_io.c
--- drivers/char/tty_io.c 1997/12/31 21:49:20 1.1
+++ drivers/char/tty_io.c 1997/12/31 21:50:59
@@ -1548,12 +1548,16 @@
switch(cmd) {
case TIOCSBRK:
case TIOCCBRK:
- return tty->driver.ioctl(tty, file, cmd, arg);
+ if (tty->driver.ioctl)
+ return tty->driver.ioctl(tty, file, cmd, arg);
+ return -EINVAL;

/* These two ioctl's always return success; even if */
/* the driver doesn't support them. */
case TCSBRK:
- case TCSBRKP:
+ case TCSBRKP:
+ if (!tty->driver.ioctl)
+ return 0;
retval = tty->driver.ioctl(tty, file, cmd, arg);
if (retval == -ENOIOCTLCMD)
retval = 0;