Re: [PATCH -next 1/8] md/raid10: prevent soft lockup while flush writes

[Yu Kuai]

Hi,

在 2023/04/25 14:39, Song Liu 写道:
> On Mon, Apr 24, 2023 at 11:16 PM Yu Kuai <yukuai1@xxxxxxxxxxxxxxx> wrote:
> > Hi,
> >
> > 在 2023/04/25 8:23, Song Liu 写道:
> > > On Thu, Apr 20, 2023 at 4:31 AM Yu Kuai <yukuai1@xxxxxxxxxxxxxxx> wrote:
> > > > From: Yu Kuai <yukuai3@xxxxxxxxxx>
> > > >
> > > > Currently, there is no limit for raid1/raid10 plugged bio. While flushing
> > > > writes, raid1 has cond_resched() while raid10 doesn't, and too many
> > > > writes can cause soft lockup.
> > > >
> > > > Follow up soft lockup can be triggered easily with writeback test for
> > > > raid10 with ramdisks:
> > > >
> > > > watchdog: BUG: soft lockup - CPU#10 stuck for 27s! [md0_raid10:1293]
> > > > Call Trace:
> > > >    <TASK>
> > > >    call_rcu+0x16/0x20
> > > >    put_object+0x41/0x80
> > > >    __delete_object+0x50/0x90
> > > >    delete_object_full+0x2b/0x40
> > > >    kmemleak_free+0x46/0xa0
> > > >    slab_free_freelist_hook.constprop.0+0xed/0x1a0
> > > >    kmem_cache_free+0xfd/0x300
> > > >    mempool_free_slab+0x1f/0x30
> > > >    mempool_free+0x3a/0x100
> > > >    bio_free+0x59/0x80
> > > >    bio_put+0xcf/0x2c0
> > > >    free_r10bio+0xbf/0xf0
> > > >    raid_end_bio_io+0x78/0xb0
> > > >    one_write_done+0x8a/0xa0
> > > >    raid10_end_write_request+0x1b4/0x430
> > > >    bio_endio+0x175/0x320
> > > >    brd_submit_bio+0x3b9/0x9b7 [brd]
> > > >    __submit_bio+0x69/0xe0
> > > >    submit_bio_noacct_nocheck+0x1e6/0x5a0
> > > >    submit_bio_noacct+0x38c/0x7e0
> > > >    flush_pending_writes+0xf0/0x240
> > > >    raid10d+0xac/0x1ed0
> > >
> > > Is it possible to trigger this with a mdadm test?
> >
> > The test I mentioned in patch 8 can trigger this problem reliablity, so
> > I this add a new test can achieve this.
>
> To be clear, by "mdadm test" I mean the tests included in mdadm:
>
> https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/tree/tests
>
> Could you please try to add a test? If it works, we should add it to
> mdadm.

Yes, of course. However, I'm not familiar how mdadm tests works yet, it
might take some time. By the way, I'll be good if I can add the test to
blktests if possible.

Thanks,
Kuai
> Thanks,
> Song
> .