x86-64: int3 no longer causes SIGTRAP in 2.6.10

[Juho Snellman]

2.6.10 changed the behaviour of the int3 instruction on x86-64. It
used to result in a SIGTRAP, now it's a SIGSEGV in both native and
32-bit legacy modes. This was apparently caused by the kprobe port,
specifically this part:

--- a/arch/x86_64/kernel/traps.c        2004-12-24 13:36:17 -08:00
+++ b/arch/x86_64/kernel/traps.c        2004-12-24 13:36:17 -08:00
@@ -862,8 +910,8 @@
        set_intr_gate(0,&divide_error);
        set_intr_gate_ist(1,&debug,DEBUG_STACK);
        set_intr_gate_ist(2,&nmi,NMI_STACK);
-       set_system_gate(3,&int3);       /* int3-5 can be called from all */
-       set_system_gate(4,&overflow);
+       set_intr_gate(3,&int3);
+       set_system_gate(4,&overflow);   /* int4-5 can be called from all */

Was effectively disabling int3 a conscious decision, or just an
unintended side-effect? This breaks at least Steel Bank Common Lisp
(x86 and x86-64) and CMU Common Lisp (x86), which use int3 for error
traps and breakpoints.

Simple test case:

% cat foo.c  
int main (void) {
    asm("int3");
    return 0;
}
% gcc -o foo foo.c
% ./foo
zsh: trace trap  ./foo
% gcc -m32 -o foo-32 foo.c
% ./foo-32
zsh: trace trap  ./foo-32

[ reboot ]

% uname -a
Linux kiki 2.6.10 #2 Sun Dec 26 04:54:05 EET 2004 x86_64 x86_64 x86_64
% ./foo
zsh: segmentation fault  ./foo
% ./foo-32
zsh: segmentation fault  ./foo-32

-- 
Juho Snellman
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/