Re: Segfault on the i386 enter instruction

[Bill Davidsen]

linux-os (Dick Johnson) wrote:
> On Fri, 12 May 2006, Tomasz Malesinski wrote:
>
> > The code attached below segfaults on the enter instruction. It works
> > when a stack frame is created by the three commented out
> > instructions and also when the first operand of the enter instruction
> > is small (less than about 6500 on my system).
> >
> > AFAIK, the only difference between creating a stack frame with the
> > enter instruction or push/mov/sub is that enter checks if the new
> > value of esp is inside the stack segment limit.
> >
> > I tested it on a vanilla kernel 2.4.26 on Intel Celeron and also on
> > probably non-vanilla 2.6.16.13 running on 3 dual core AMD Opteron,
> > quite busy, server. It is working in 32-bit mode. Interestingly, on
> > the second machine sometimes the program worked correctly.
> >
> > I am not subscribed to the list. Please cc replies to me.
> >
> >
> > 	.file	"a.c"
> > 	.version	"01.01"
> > gcc2_compiled.:
> > .section	.rodata
> > .LC0:
> > 	.string	"asdf\n"
> > .text
> > 	.align 4
> > .globl main
> > 	.type	 main,@function
> > main:
> > 	enter $10008, $0
> > #	pushl %ebp
> > #	movl %esp,%ebp
> > #	subl $10008,%esp
> > 	addl $-12,%esp
>          ^^^^^^^^^^^^^^____________ WTF
>          adding a negative number is subtracting that positive value.
Right, adding -12 is the same as subtracting 12. I have no idea what 
you're getting at with the next two lines.
>          You just subtracted 0xfffffff3 (on a 32-bit machine) from
>          the stack pointer. It damn-well better seg-fault!
No, we subtracted 12. I'm not sure where that number came from, it's the 
1's complement of 12 but I'm dead sure Linux code isn't running on any 
1's comp machines.

--
   -bill davidsen (davidsen@xxxxxxx)
"The secret to procrastination is to put things off until the
 last possible moment - but no longer"  -me
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/