Re: [PATCH] block: make sure big bio is splitted into at most 256 bvecs

From: Ming Lei
Date: Thu Apr 07 2016 - 20:21:27 EST

On Fri, Apr 8, 2016 at 7:29 AM, Eric Wheeler <bcache@xxxxxxxxxxxxxxxxxx> wrote:
> On Thu, 7 Apr 2016, Ming Lei wrote:
>
>> On Thu, Apr 7, 2016 at 9:56 AM, Eric Wheeler <bcache@xxxxxxxxxxxxxxxxxx> wrote:
>> > On Thu, 7 Apr 2016, Ming Lei wrote:
>> >
>> >> On Thu, Apr 7, 2016 at 9:36 AM, Eric Wheeler <bcache@xxxxxxxxxxxxxxxxxx> wrote:
>> >> > On Wed, 6 Apr 2016, Ming Lei wrote:
>> >> >
>> >> >> After arbitrary bio size is supported, the incoming bio may
>> >> >> be very big. We have to split the bio into small bios so that
>> >> >> each holds at most BIO_MAX_PAGES bvecs for safety reason, such
>> >> >> as bio_clone().
>> >> >>
>> >> >> This patch fixes the following kernel crash:
>> >> >>
>> >> >> > [ 172.660142] BUG: unable to handle kernel NULL pointer dereference at
>> >> >> > 0000000000000028
>> >> >> > [ 172.660229] IP: [<ffffffff811e53b4>] bio_trim+0xf/0x2a
>> >> >> > [ 172.660289] PGD 7faf3e067 PUD 7f9279067 PMD 0
>> >> >> > [ 172.660399] Oops: 0000 [#1] SMP
>> >> >> > [...]
>> >> >> > [ 172.664780] Call Trace:
>> >> >> > [ 172.664813] [<ffffffffa007f3be>] ? raid1_make_request+0x2e8/0xad7 [raid1]
>> >> >> > [ 172.664846] [<ffffffff811f07da>] ? blk_queue_split+0x377/0x3d4
>> >> >> > [ 172.664880] [<ffffffffa005fb5f>] ? md_make_request+0xf6/0x1e9 [md_mod]
>> >> >> > [ 172.664912] [<ffffffff811eb860>] ? generic_make_request+0xb5/0x155
>> >> >> > [ 172.664947] [<ffffffffa0445c89>] ? prio_io+0x85/0x95 [bcache]
>> >> >> > [ 172.664981] [<ffffffffa0448252>] ? register_cache_set+0x355/0x8d0 [bcache]
>> >> >> > [ 172.665016] [<ffffffffa04497d3>] ? register_bcache+0x1006/0x1174 [bcache]
>> >> >>
>> >> >> Fixes: 54efd50(block: make generic_make_request handle arbitrarily sized bios)
>> >> >> Reported-by: Sebastian Roesner <sroesner-kernelorg@xxxxxxxxxxxxxxxxx>
>> >> >> Reported-by: Eric Wheeler <bcache@xxxxxxxxxxxxxxxxxx>
>> >> >> Cc: stable@xxxxxxxxxxxxxxx (4.2+)
>> >> >
>> >> > Ming Lei,
>> >> >
>> >> > get_max_io_size doesn't appear until 4.5 based on a quick LXR check so we
>> >> > won't see it in stable I don't think.
>> >> >
>> >> > It would be nice to see this fixed in 4.1 (if affected there). Is there
>> >>
>> >> The issue should be introduced to v4.3 via 54efd50
>> >>
>> >> > another place this could be applied to be a bit more backward compatible?
>> >>
>> >> The v1 needn't change to get_max_io_size(), and it should be simple enough
>> >> to backport to previous stables, please try it:
>> >>
>> >> http://marc.info/?l=linux-block&m=145991422422927&w=2
>> >
>> > V1 changes blk_bio_segment_split() which doesn't exist until 4.3.
>> >
>> > How might you port this to v4.1.y?
>>
>> Can you see the issue with v4.1?
>>
>> You mentioned there are three reports:
>>
>> > [2014-02-04] https://bugzilla.redhat.com/show_bug.cgi?id=1061339
>> > [2016-01-13] http://www.spinics.net/lists/linux-bcache/msg03335.html
>> https://bugzilla.kernel.org/show_bug.cgi?id=110771
>> > [2016-03-25] http://thread.gmane.org/gmane.linux.kernel.bcache.devel/3607 [this thread]
>> https://bugzilla.kernel.org/show_bug.cgi?id=114871
>>
>> The first one has been fixed by '8423ae3 block: Fix cloning of
>> discard/write same bios', as mentioned in bugzilla.
>>
>> The other two are reported on v4.4 and v4.5.
>>
>> If you look at the patch description, it is just needed for 4.3+.
>
>
> Oh, that could be---but just to be sure:
>
> I had thought perhaps this was an old issue since the first mention of
> this backtrace (but not bcache) was in 3.14 back in 2014 based on this
> post:
> https://bugzilla.redhat.com/show_bug.cgi?id=1061339
>
> Is this relevant?

Eric,

That is another bio_clone() issue, which was fixed by commit
'8423ae3 block: Fix cloning of discard/write same bios'.

If you did read the bugzillar comment and my reply, you should
get the answer already.

I am sorry for having to answer your same question again and again...

>
> --
> Eric Wheeler
>
>
>
>>
>> Or I am wrong?
>>
>>
>> Thanks,
>>
>> >
>> > --
>> > Eric Wheeler
>> >
>> >
>> >>
>> >> Thanks,
>> >>
>> >> >
>> >> >> Cc: Shaohua Li <shli@xxxxxx>
>> >> >> Signed-off-by: Ming Lei <ming.lei@xxxxxxxxxxxxx>
>> >> >> ---
>> >> >> I can reproduce the issue and verify the fix by the following approach:
>> >> >> - create one raid1 over two virtio-blk
>> >> >> - build bcache device over the above raid1 and another cache device.
>> >> >> - set cache mode as writeback
>> >> >> - run random write over ext4 on the bcache device
>> >> >> - then the crash can be triggered
>> >> >>
>> >> >> block/blk-merge.c | 12 ++++++++++++
>> >> >> 1 file changed, 12 insertions(+)
>> >> >>
>> >> >> diff --git a/block/blk-merge.c b/block/blk-merge.c
>> >> >> index 2613531..9a8651f 100644
>> >> >> --- a/block/blk-merge.c
>> >> >> +++ b/block/blk-merge.c
>> >> >> @@ -79,6 +79,18 @@ static inline unsigned get_max_io_size(struct request_queue *q,
>> >> >> /* aligned to logical block size */
>> >> >> sectors &= ~(mask >> 9);
>> >> >>
>> >> >> + /*
>> >> >> + * With arbitrary bio size, the incoming bio may be very big.
>> >> >> + * We have to split the bio into small bios so that each holds
>> >> >> + * at most BIO_MAX_PAGES bvecs for safety reason, such as
>> >> >> + * bio_clone().
>> >> >> + *
>> >> >> + * In the future, the limit might be converted into per-queue
>> >> >> + * flag.
>> >> >> + */
>> >> >> + sectors = min_t(unsigned, sectors, BIO_MAX_PAGES <<
>> >> >> + (PAGE_CACHE_SHIFT - 9));
>> >> >> +
>> >> >> return sectors;
>> >> >> }
>> >> >>
>> >> >> --
>> >> >> 1.9.1
>> >> >>
>> >> >>
>> >> > --
>> >> > To unsubscribe from this list: send the line "unsubscribe linux-block" in
>> >> > the body of a message to majordomo@xxxxxxxxxxxxxxx
>> >> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>> >>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-block" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html