Re: bcache: Fix a writeback performance regression

From: Stefan Priebe
Date: Thu Aug 22 2013 - 01:25:24 EST


Am 22.08.2013 01:47, schrieb Kent Overstreet:
On Tue, Aug 20, 2013 at 10:07:45AM +0200, Stefan Priebe - Profihost AG wrote:
Am 20.08.2013 10:01, schrieb Stefan Priebe - Profihost AG:
Am 20.08.2013 00:27, schrieb Kent Overstreet:
On Mon, Aug 19, 2013 at 12:09:24AM +0200, Stefan Priebe wrote:

Vanilla 3.10.7 + bcache: Fix a writeback performance regression

http://pastebin.com/raw.php?i=LXZk4cMH

Whoops, at first I thought this was the same bug as one I'd already been
chasing down that had been a harmless bug - turns out I didn't look
closely enough at the backtrace.

What happened is background writeback is deadlocking, because for some
reason the workqueue it's running out of is a singlethreaded workqueue,
so as soon as it decides to queue enough writeback bios that it has to
sleep on that semaphore (which often won't happen due to the PD
controller based ratelimiting) - boom, deadlock.

Here's the fixup patch I just tested and am applying:

Oh i'm now seeing very high CPU spikes of kworker... i don't see if i
remove bcache: Fix a writeback performance regression.

Are you able to reproduce it? I'm not having any luck reproducing it...

Hi,

yes but only on production system it seems it happens when there is a big bunch in dirty and having heavy 4K random I/O.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/