Re: PROBLEM: kernel BUG at mm/slab.c:3002!
From: Alan D. Brunelle
Date: Tue Mar 10 2009 - 13:14:31 EST
Alan D. Brunelle wrote:
> Running blktrace & I/O loads cause a kernel BUG at mm/slab.c:3002!.
>
> I'm running some moderate I/O loads (using 12 devices behind a Smart
> Array on a 16-way x86_64 box, I'm doing asynchronous direct sequential
> reads to each disk in parallel), and whilst attempting to get blktrace
> data I routinely run into this.
>
> I first started seeing this in 2.6.29-rc6, so I bumped to 2.6.29-rc7 and
> made a couple of successful runs, then ran into it again. (I've
> attached the script I was using, but I'm not sure it's very helpful...)
>
> The environment the system under test is in is rather difficult to
> bisect in, but if need be, I can certainly go through the (painful)
> motions to do so...
>
> I only ran this a couple of times on 2.6.27.7-4 (SLESS 11 b6 kernel),
> and both times it worked there - not sure how far back the problem occurs...
>
> I'm open to any SLAB debug tracing options that may help with this...
>
> Alan D.Brunelle
> Hewlett-Packard
>
Hm, it isn't SLAB-related: reconfigured w/ SLUB and got:
BUG: unable to handle kernel NULL pointer dereference at 0000000000000030g
IP: [<ffffffff802d0fda>] deactivate_super+0x11/0x8cg
PGD 1867047067 PUD 1853074067 PMD 0 g
Oops: 0000 [#1] SMP g
last sysfs file:
/sys/devices/pci0000:40/0000:40:10.0/0000:41:02.0/local_cpusg
CPU 15 g
Modules linked in: xfs exportfs fuse ext2 loop dm_mod sd_mod crc_t10dif
qla2xxx sg bnx2 scsi_transport_fc rtc_cmos sr_mod ipmi_si rtc_core
pcspkr ipmi_msghandler hpwdt hpilo container shpchp serio_raw scsi_tgt
rtc_lib button cdrom pci_hotplug usbhid hid uhci_hcd ohci_hcd ehci_hcd
usbcore edd ext3 mbcache jbd fan ide_pci_generic amd74xx ide_core
pata_amd thermal processor thermal_sys hwmon cciss ata_generic libata
scsi_modg
Pid: 12527, comm: blktrace Not tainted 2.6.29-rc7 #4 ProLiant DL585 G5 g
RIP: 0010:[<ffffffff802d0fda>] [<ffffffff802d0fda>]
deactivate_super+0x11/0x8cg
RSP: 0018:ffff88087a4e1b38 EFLAGS: 00010246g
RAX: ffff881829debf00 RBX: 0000000000000000 RCX: 00000000fffffffeg
RDX: 0000000000000d74 RSI: ffffffff80ab4b00 RDI: 0000000000000000g
RBP: ffff88087a4e1b48 R08: 0000000000000000 R09: 0000000000000000g
R10: ffff88107cc3d760 R11: ffff88187b84deb8 R12: ffff881829debf00g
R13: 0000000000000200 R14: ffffffff809a7d00 R15: ffff881829debfd8g
FS: 00007fab4778f6f0(0000) GS:ffff88207c202380(0000)
knlGS:0000000000000000g
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003bg
CR2: 0000000000000030 CR3: 0000001801474000 CR4: 00000000000006e0g
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000g
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400g
Process blktrace (pid: 12527, threadinfo ffff88087a4e0000, task
ffff880821e70000)g
Stack:g
ffff88189c255d00 ffff881829debf00 ffff88087a4e1b88 ffffffff802e4f22g
0000000000000000 ffffffff80ab82b8 ffffffff80ab82b0 ffff881829debf00g
ffff8820798f6148 0000000000000016 ffff88087a4e1bb8 ffffffff802e8faeg
Call Trace:g
[<ffffffff802e4f22>] mntput_no_expire+0x144/0x18fg
[<ffffffff802e8fae>] simple_release_fs+0x68/0x70g
[<ffffffff8032cd10>] debugfs_remove+0x5b/0x60g
[<ffffffff80289c02>] ? relay_remove_buf+0x0/0x2cg
[<ffffffff8036c255>] blk_remove_buf_file_callback+0x1a/0x20g
[<ffffffff80289c21>] relay_remove_buf+0x1f/0x2cg
[<ffffffff8037155d>] kref_put+0x4b/0x57g
[<ffffffff80289c85>] relay_close_buf+0x35/0x3ag
[<ffffffff8028a08c>] relay_close+0x5e/0xecg
[<ffffffff8036ca9f>] blk_trace_remove+0x50/0x1e3g
[<ffffffff8036cd0f>] blk_trace_ioctl+0xb3/0xcfg
[<ffffffffa03aa4c5>] ? xfs_write+0x6c8/0x6e3 [xfs]g
[<ffffffff80363d78>] blkdev_ioctl+0x803/0x853g
[<ffffffff804b1100>] ? _spin_lock+0x17/0x1ag
[<ffffffff803702ac>] ? kobject_put+0x47/0x4bg
[<ffffffff803f6e38>] ? put_device+0x15/0x17g
[<ffffffffa000017a>] ? scsi_device_put+0x3d/0x42 [scsi_mod]g
[<ffffffffa0312bcb>] ? scsi_disk_put+0x3a/0x3f [sd_mod]g
[<ffffffff802f30a3>] block_ioctl+0x38/0x3cg
[<ffffffff802db638>] vfs_ioctl+0x2a/0x78g
[<ffffffff802dbacc>] do_vfs_ioctl+0x446/0x482g
[<ffffffff802cfc80>] ? __fput+0x18c/0x199g
[<ffffffff802dbb5d>] sys_ioctl+0x55/0x77g
[<ffffffff8020c42a>] system_call_fastpath+0x16/0x1bg
Code: 48 8b 7f 68 48 85 ff 74 05 e8 45 cb 00 00 48 89 df e8 8d ff ff ff
5b 5b c9 c3 55 48 c7 c6 00 4b ab 80 48 89 e5 41 54 53 48 89 fb <4c> 8b
67 30 48 8d bf a8 00 00 00 e8 0a de 09 00 85 c0 74 62 81 g
RIP [<ffffffff802d0fda>] deactivate_super+0x11/0x8cg
RSP <ffff88087a4e1b38>g
CR2: 0000000000000030g
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/