Re: general protection fault in process_one_work

[Ido Schimmel]

On Tue, Oct 29, 2019 at 10:45:19AM +0200, Ido Schimmel wrote:
> On Tue, Oct 29, 2019 at 09:43:27AM +0100, Dmitry Vyukov wrote:
> > On Tue, Oct 29, 2019 at 9:38 AM syzbot
> > <syzbot+9ed8f68ab30761f3678e@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote:
> > >
> > > Hello,
> > >
> > > syzbot found the following crash on:
> > >
> > > HEAD commit:    38207291 bpf: Prepare btf_ctx_access for non raw_tp use case
> > > git tree:       bpf-next
> > > console output: https://syzkaller.appspot.com/x/log.txt?x=14173c0f600000
> > > kernel config:  https://syzkaller.appspot.com/x/.config?x=41648156aa09be10
> > > dashboard link: https://syzkaller.appspot.com/bug?extid=9ed8f68ab30761f3678e
> > > compiler:       gcc (GCC) 9.0.0 20181231 (experimental)
> > >
> > > Unfortunately, I don't have any reproducer for this crash yet.
> > >
> > > IMPORTANT: if you fix the bug, please add the following tag to the commit:
> > > Reported-by: syzbot+9ed8f68ab30761f3678e@xxxxxxxxxxxxxxxxxxxxxxxxx
> > 
> > +JiÅÃ PÃrko, this seems to be related to netdevsim.
> 
> Will check it.

Didn't have a lot of time today, but I think the issue is a race
condition (note that syzbot only triggered it twice so far).

Upon reload nsim_dev_port_del_all() is called and starts deleting ports
from the ports list without holding the ports list mutex. It is possible
that during this time nsim_dev_trap_report_work() is executing from a
workqueue and accessing freed memory despite holding the ports list
mutex.

I'll try to reproduce and send a fix later this week.

> 
> > 
> > > kasan: CONFIG_KASAN_INLINE enabled
> > > kasan: GPF could be caused by NULL-ptr deref or user memory access
> > > general protection fault: 0000 [#1] PREEMPT SMP KASAN
> > > CPU: 1 PID: 9149 Comm: kworker/1:3 Not tainted 5.4.0-rc1+ #0
> > > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
> > > Google 01/01/2011
> > > Workqueue: events nsim_dev_trap_report_work
> > > RIP: 0010:nsim_dev_trap_report_work+0xc4/0xaf0
> > > drivers/net/netdevsim/dev.c:409
> > > Code: 89 45 d0 0f 84 8b 07 00 00 49 bc 00 00 00 00 00 fc ff df e8 3e ae ef
> > > fc 48 8b 45 d0 48 05 68 01 00 00 48 89 45 90 48 c1 e8 03 <42> 80 3c 20 00
> > > 0f 85 b1 09 00 00 48 8b 45 d0 48 8b 98 68 01 00 00
> > > RSP: 0018:ffff88806c98fc90 EFLAGS: 00010a06
> > > RAX: 1bd5a0000000004d RBX: 0000000000000000 RCX: ffffffff84836e22
> > > RDX: 0000000000000000 RSI: ffffffff84836db2 RDI: 0000000000000001
> > > RBP: ffff88806c98fd30 R08: ffff88806c9863c0 R09: ffffed100d75f3d9
> > > R10: ffffed100d75f3d8 R11: ffff88806baf9ec7 R12: dffffc0000000000
> > > R13: ffff88806baf9ec0 R14: ffff8880a9a13900 R15: ffff8880ae934500
> > > FS:  0000000000000000(0000) GS:ffff8880ae900000(0000) knlGS:0000000000000000
> > > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > CR2: 00007efdd0c9e000 CR3: 000000009cc1b000 CR4: 00000000001406e0
> > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > > Call Trace:
> > >   process_one_work+0x9af/0x1740 kernel/workqueue.c:2269
> > >   worker_thread+0x98/0xe40 kernel/workqueue.c:2415
> > >   kthread+0x361/0x430 kernel/kthread.c:255
> > >   ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
> > > Modules linked in:
> > > ---[ end trace ba29cd1c27f63d86 ]---
> > > RIP: 0010:nsim_dev_trap_report_work+0xc4/0xaf0
> > > drivers/net/netdevsim/dev.c:409
> > > Code: 89 45 d0 0f 84 8b 07 00 00 49 bc 00 00 00 00 00 fc ff df e8 3e ae ef
> > > fc 48 8b 45 d0 48 05 68 01 00 00 48 89 45 90 48 c1 e8 03 <42> 80 3c 20 00
> > > 0f 85 b1 09 00 00 48 8b 45 d0 48 8b 98 68 01 00 00
> > > RSP: 0018:ffff88806c98fc90 EFLAGS: 00010a06
> > > RAX: 1bd5a0000000004d RBX: 0000000000000000 RCX: ffffffff84836e22
> > > RDX: 0000000000000000 RSI: ffffffff84836db2 RDI: 0000000000000001
> > > RBP: ffff88806c98fd30 R08: ffff88806c9863c0 R09: ffffed100d75f3d9
> > > R10: ffffed100d75f3d8 R11: ffff88806baf9ec7 R12: dffffc0000000000
> > > R13: ffff88806baf9ec0 R14: ffff8880a9a13900 R15: ffff8880ae934500
> > > FS:  0000000000000000(0000) GS:ffff8880ae900000(0000) knlGS:0000000000000000
> > > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > CR2: 00007efdd0c9e000 CR3: 000000009cc1b000 CR4: 00000000001406e0
> > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > >
> > >
> > > ---
> > > This bug is generated by a bot. It may contain errors.
> > > See https://goo.gl/tpsmEJ for more information about syzbot.
> > > syzbot engineers can be reached at syzkaller@xxxxxxxxxxxxxxxxx
> > >
> > > syzbot will keep track of this bug report. See:
> > > https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bugs+unsubscribe@xxxxxxxxxxxxxxxxx
> > > To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/0000000000001c46d5059608892f%40google.com.