Re: Patch "x86: Store a per-cpu shadow copy of CR4" gives VirtualBox 4.3.24 indigestion.

[Andy Lutomirski]

On Tue, Mar 3, 2015 at 4:49 PM, Valdis Kletnieks
<Valdis.Kletnieks@xxxxxx> wrote:
> Just a heads up, as it's a out-of-tree module, so this is mostly to
> get it out for the benefit of others googling for this..
>
> I'm seeing a kernel panic with VirtualBox 4.3.24 with a weird but consistent traceback:
>
> [  359.370165] created IOMMU domain ffff8801ffeecd80
> [  395.739820] general protection fault: 0000 [#1] PREEMPT SMP
>
> [  395.739917] task: ffff88022cab4310 ti: ffff88022cab8000 task.ti: ffff88022cab8000
> [  395.739921] RIP: 0010:[<ffffffffa803cd02>]  [<ffffffffa803cd02>] do_flush_tlb_all+0x26/0x4f
> [  395.739932] RSP: 0018:ffff88022dd03f68  EFLAGS: 00010006
> [  395.739935] RAX: 00000000001407e0 RBX: ffff88022dd1ec80 RCX: 0000000000140760
> [  395.739939] RDX: 0000000000000002 RSI: 0000000000000000 RDI: 0000000000000000
> [  395.739942] RBP: ffff88022dd03f68 R08: 0000000000000001 R09: 0000000000000001
> [  395.739946] R10: ffff88022cab4310 R11: ffffffffffffffdb R12: ffff88022dd15580
> [  395.739949] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000014500
> [  395.739954] FS:  0000000000000000(0000) GS:ffff88022dd00000(0000) knlGS:0000000000000000
> [  395.739957] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [  395.739961] CR2: 0000000094e7de6f CR3: 0000000028c0b000 CR4: 00000000001427e0
> [  395.739964] Stack:
> [  395.739967]  ffff88022dd03f98 ffffffffa80b83d6 ffffe8ffffd08ed8 0000000000000003
> [  395.739976]  0000005c037d19a5 0000005c037dfc49 ffff88022dd03fa8 ffffffffa80297bb
> [  395.739985]  ffff88022cabbe88 ffffffffa871de70 ffff88022cabbdc8 <EOI>  ffff88022cabbe88
> [  395.739996] Call Trace:
> [  395.740000]  <IRQ>
> [  395.740003]  [<ffffffffa80b83d6>] generic_smp_call_function_single_interrupt+0x108/0x1ca
> [  395.740018]  [<ffffffffa80297bb>] smp_call_function_interrupt+0x18/0x26
> [  395.740025]  [<ffffffffa871de70>] call_function_interrupt+0x70/0x80
> [  395.740028]  <EOI>
> [  395.740031]  [<ffffffffa8085e8d>] ? trace_hardirqs_off_caller+0x1f/0xac
> [  395.740046]  [<ffffffffa84a23fa>] ? cpuidle_enter_state+0x60/0xb3
> [  395.740052]  [<ffffffffa80885fb>] ? trace_hardirqs_on+0xd/0xf
> [  395.740058]  [<ffffffffa84a23fe>] ? cpuidle_enter_state+0x64/0xb3
> [  395.740065]  [<ffffffffa84a23fa>] ? cpuidle_enter_state+0x60/0xb3
> [  395.740071]  [<ffffffffa84a2479>] cpuidle_enter+0x17/0x19
> [  395.740078]  [<ffffffffa8080853>] cpu_startup_entry+0x286/0x3e7
> [  395.740084]  [<ffffffffa80b0119>] ? clockevents_register_device+0x1e6/0x207
> [  395.740091]  [<ffffffffa802a168>] start_secondary+0x14e/0x17b
> [  395.740097] Code: 00 eb d9 c3 c3 0f 1f 44 00 00 48 8b 05 ec 24 d2 00 55 48 89 e5 f6 c4 20 74 1b 9c 5a fa 65 48 8b 05 54 6d fd 57 48 89 c1 80 e1 7f <0f> 22 e1 0f 22 e0 52 9d eb 05 e8 da fb ff ff 65 8b 05 30 6d fd
>
> Bisection points at:
>
> commit 1e02ce4cccdcb9688386e5b8d2c9fa4660b45389
> Author: Andy Lutomirski <luto@xxxxxxxxxxxxxx>
> Date:   Fri Oct 24 15:58:08 2014 -0700
>
>     x86: Store a per-cpu shadow copy of CR4
>
>     Context switches and TLB flushes can change individual bits of CR4.
>     CR4 reads take several cycles, so store a shadow copy of CR4 in a
>     per-cpu variable.
>
> I'm reasonably sure that the problem is that VirtualBox isn't onboard
> with the whole shadow copy thing, and hilarity and hijinks result when
> it twiddles CR4 and doesn't update the shadow copy, and they get out of
> sync.
>
> Am also poking the VirtualBox people.

Are they poking cr4 directly with mov?  Let's see:

    /* Enable the VMX bit in CR4 if necessary. */
    RTCCUINTREG uCr4 = ASMGetCR4();
    if (!(uCr4 & X86_CR4_VMXE))
        ASMSetCR4(uCr4 | X86_CR4_VMXE);

ASMSetCR4 is about what you'd expect.

I deliberately renamed write_cr4 to __write_cr4 to catch this kind of
thing in anything that uses the asm/ headers, but I can't do anything
about asm code in out-of-tree modules.

--Andy

-- 
Andy Lutomirski
AMA Capital Management, LLC
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/