Re: [PATCH 00/46] rcu-walk and dcache scaling

From: Peter Zijlstra
Date: Tue Dec 07 2010 - 11:23:46 EST


On Wed, 2010-12-08 at 02:59 +1100, Nick Piggin wrote:
> OK, thanks. My version should not have caused any problems though,
> right?

I tihnk so, yes, altough looking at it again I wonder why you use
spin_aquire(.trylock=1) -- but that too shouldn't cause anything like
the explosion.

> Any idea what might have caused Dave's crash?

Not directly, no. Usually lockdep crashes indicate use after free like
things, where we try to lock a lock that's been scribbled on. But that
usually explodes a bit earlier.

You faulting in the middle of that breath-first-search does suggest some
data corruption, but I'm not quite sure what kind, I can't remember ever
having seem something like this before.

I've CC'ed Ming Lei who wrote the bfs search, maybe he's got an idea.


Copy of the splat:
---

> [ 5954.061633] BUG: sleeping function called from invalid context at arch/x86/mm/fault.c:1081
> [ 5954.062466] in_atomic(): 0, irqs_disabled(): 1, pid: 2927, name: rm
> [ 5954.063122] 3 locks held by rm/2927:
> [ 5954.063476] #0: (&sb->s_type->i_mutex_key#12/1){+.+.+.}, at: [<ffffffff8116f5e1>] do_rmdir+0x81/0x130
> [ 5954.064014] #1: (&sb->s_type->i_mutex_key#12){+.+.+.}, at: [<ffffffff8116d3a8>] vfs_rmdir+0x58/0xe0
> [ 5954.064014] #2: (rcu_read_lock){.+.+..}, at: [<ffffffff811779c0>] shrink_dentry_list+0x0/0x430
> [ 5954.064014] irq event stamp: 1484376719
> [ 5954.064014] hardirqs last enabled at (1484376719): [<ffffffff810ebf07>] __call_rcu+0xd7/0x1a0
> [ 5954.064014] hardirqs last disabled at (1484376718): [<ffffffff810ebe7a>] __call_rcu+0x4a/0x1a0
> [ 5954.064014] softirqs last enabled at (1484376586): [<ffffffff8108b911>] __do_softirq+0x161/0x270
> [ 5954.064014] softirqs last disabled at (1484376581): [<ffffffff8103af1c>] call_softirq+0x1c/0x50
> [ 5954.064014] Pid: 2927, comm: rm Not tainted 2.6.37-rc4-dgc+ #794
> [ 5954.064014] Call Trace:
> [ 5954.064014] [<ffffffff810b95b0>] ? print_irqtrace_events+0xd0/0xe0
> [ 5954.064014] [<ffffffff81076455>] __might_sleep+0xf5/0x130
> [ 5954.064014] [<ffffffff81b1e603>] do_page_fault+0x103/0x4f0
> [ 5954.064014] [<ffffffff810645b8>] ? pvclock_clocksource_read+0x58/0xd0
> [ 5954.064014] [<ffffffff810645b8>] ? pvclock_clocksource_read+0x58/0xd0
> [ 5954.064014] [<ffffffff810645b8>] ? pvclock_clocksource_read+0x58/0xd0
> [ 5954.064014] [<ffffffff81b19b28>] ? trace_hardirqs_off_thunk+0x3a/0x3c
> [ 5954.064014] [<ffffffff810b8e20>] ? usage_match+0x0/0x20
> [ 5954.064014] [<ffffffff81b1af25>] page_fault+0x25/0x30
> [ 5954.064014] [<ffffffff810b8e20>] ? usage_match+0x0/0x20
> [ 5954.064014] [<ffffffff810ba1c8>] ? __bfs+0xc8/0x260
> [ 5954.064014] [<ffffffff810ba123>] ? __bfs+0x23/0x260
> [ 5954.064014] [<ffffffff810ba4d2>] find_usage_backwards+0x42/0x80
> [ 5954.064014] [<ffffffff810bcec4>] check_usage_backwards+0x64/0xf0
> [ 5954.064014] [<ffffffff8104796f>] ? save_stack_trace+0x2f/0x50
> [ 5954.064014] [<ffffffff810bce60>] ? check_usage_backwards+0x0/0xf0
> [ 5954.064014] [<ffffffff810bd9a9>] mark_lock+0x1a9/0x440
> [ 5954.064014] [<ffffffff810be989>] __lock_acquire+0x5a9/0x14b0
> [ 5954.064014] [<ffffffff810be716>] ? __lock_acquire+0x336/0x14b0
> [ 5954.064014] [<ffffffff810645b8>] ? pvclock_clocksource_read+0x58/0xd0
> [ 5954.064014] [<ffffffff810bf944>] lock_acquire+0xb4/0x140
> [ 5954.064014] [<ffffffff81177a1c>] ? shrink_dentry_list+0x5c/0x430
> [ 5954.064014] [<ffffffff81b19d86>] _raw_spin_lock+0x36/0x70
> [ 5954.064014] [<ffffffff81177a1c>] ? shrink_dentry_list+0x5c/0x430
> [ 5954.064014] [<ffffffff81177a1c>] shrink_dentry_list+0x5c/0x430
> [ 5954.064014] [<ffffffff811779c0>] ? shrink_dentry_list+0x0/0x430
> [ 5954.064014] [<ffffffff816b9c7e>] ? do_raw_spin_unlock+0x5e/0xb0
> [ 5954.064014] [<ffffffff81177f2d>] __shrink_dcache_sb+0x13d/0x1c0
> [ 5954.064014] [<ffffffff811784bf>] shrink_dcache_parent+0x32f/0x390
> [ 5954.064014] [<ffffffff8116d31d>] dentry_unhash+0x3d/0x70
> [ 5954.064014] [<ffffffff8116d3b0>] vfs_rmdir+0x60/0xe0
> [ 5954.064014] [<ffffffff8116f673>] do_rmdir+0x113/0x130
> [ 5954.064014] [<ffffffff8103a03a>] ? sysret_check+0x2e/0x69
> [ 5954.064014] [<ffffffff81b19ae9>] ? trace_hardirqs_on_thunk+0x3a/0x3f
> [ 5954.064014] [<ffffffff8116f6c5>] sys_unlinkat+0x35/0x40
> [ 5954.064014] [<ffffffff8103a002>] system_call_fastpath+0x16/0x1b
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/