Re: [patch]block: revert a patch

[Williams, Dan J]

On Thu, Aug 11, 2011 at 12:11 AM, Shaohua Li <shaohua.li@xxxxxxxxx> wrote:
> This patch reverts commit 35ae66e0a09ab70ed(block: Make rq_affinity = 1
> work as expected). The purpose is to avoid an unnecessary IPI.
> Let's take an example. My test box has cpu 0-7, one socket. Say request is
> added from CPU 1, blk_complete_request() occurs at CPU 7. Without the reverted
> patch, softirq will be done at CPU 7. With it, an IPI will be directed to CPU
> 0, and softirq will be done at CPU 0. In this case, doing softirq at CPU 0 and
> CPU 7 have no difference from cache sharing point view and we can avoid an
> ipi if doing it in CPU 7.
> An immediate concern is this is just like QUEUE_FLAG_SAME_FORCE, but actually
> not. blk_complete_request() is running in interrupt handler, and currently
> I/O controller doesn't support multiple interrupts (I checked several LSI
> cards and AHCI), so only one CPU can run blk_complete_request(). This is
> still quite different as QUEUE_FLAG_SAME_FORCE.
> Since only one CPU runs softirq, the only difference with below patch is
> softirq not always runs at the first CPU of a group.

Ah, so I misinterpreted the initial implementation in a beneficial way.

I suspect this might be the real root cause of the iops bump in the
rq_affinity=1 case that we saw when testing this patch set [1].
Because the hack [2] of looking at the state of ksoftirqd on 'ccpu'
missed the 'waking' state of ksoftirqd, and when that was added
performance did not improve it actually went down a bit.

--
Dan

[1]: https://lkml.org/lkml/2011/7/22/294
[2]: https://lkml.org/lkml/2011/7/22/296
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/