Re: rcu_prempt stalls / lockup

From: Dave Jones
Date: Thu Apr 03 2014 - 16:02:20 EST


On Wed, Apr 02, 2014 at 06:48:40PM -0400, Dave Jones wrote:

> > > > > Waiting uninterruptibly. Presumably blocked on mutex_lock(). But
> > > > > you have CONFIG_PROVE_LOCKING(), so any deadlocks should have been
> > > > > reported.
> > > >
> > > > Lockdep had reported something a little earlier (timestamped at 1108.xxxxxx)
> > > > but that's a known false-positive in xfs.
> > >
> > > Yep, I would be very surprised if that was related to the grace-period hang.
> >
> > Ah, but it could be suppressing later lockdep splats. So if this can be
> > reproduced without xfs, we might get additional information from lockdep.
>
> Hrmph.
>
> $ git bisect bad
> The merge base 5cb480f6b488128140c940abff3c36f524a334a8 is bad.
> This means the bug has been fixed between 5cb480f6b488128140c940abff3c36f524a334a8 and [455c6fdbd219161bd09b1165f11699d6d73de11c 62c206bd514600d4d73751ade00dca8e488390a3 e086481baf9d0436bdd6e9b739bfa4a83fb89ef5].
>
> Not sure where to go from here..
>
> The 'good' news is I can reproduce it pretty reliably now.
> I start my fuzz tester, and immediately do a git diff in my working tree,
> and then boom..

Even better, now I realise I don't even need my fuzzer in the mix. Just doing
a fair amount of disk io (like a git diff on a dirty tree) will trigger it.

I've tried adding a show_state() call when the stall happens, but another stall
seems to occur before it gets a chance to even dump everything over the usb-serial console.
And of course nothing ever makes it to disk, even though I can sysrq-sync, on the
next reboot systemd has stuffed a bunch of ^@ in the log where the interesting
stuff should be.

Any other ideas ?

Dave


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/