Re: [PATCH 5/7] blk-mq: remove REQ_ATOM_COMPLETE usages from blk-mq

From: Tejun Heo
Date: Mon Jan 08 2018 - 12:27:33 EST


Hello, Jianchao.

On Fri, Dec 22, 2017 at 12:02:20PM +0800, jianchao.wang wrote:
> > On Thu, Dec 21, 2017 at 11:56:49AM +0800, jianchao.wang wrote:
> >> It's worrying that even though the blk_mark_rq_complete() here is
> >> intended to synchronize with timeout path, but it indeed give the
> >> blk_mq_complete_request() the capability to exclude with
>
> There could be scenario where the driver itself stop a request
> itself with blk_mq_complete_request() or some other interface that
> will invoke it, races with the normal completion path where a same
> request comes.

But what'd prevent the completion reinitializing the request and then
the actual completion path coming in and completing the request again?

> For example:
> a reset could be triggered through sysfs on nvme-rdma
> Then the driver will cancel all the reqs, including in-flight ones.
> nvme_rdma_reset_ctrl_work()
> nvme_rdma_shutdown_ctrl()
> >>>>
> if (ctrl->ctrl.queue_count > 1) {
> nvme_stop_queues(&ctrl->ctrl); //quiesce the queue
> blk_mq_tagset_busy_iter(&ctrl->tag_set,
> nvme_cancel_request, &ctrl->ctrl); //invoke blk_mq_complete_request()
> nvme_rdma_destroy_io_queues(ctrl, shutdown);
> }
> >>>>
>
> These operations could race with the normal completion path of in-flight ones.
> It should drain all the in-flight ones first here. But there maybe some other
> places similar with this.

If there are any such places, they should be using an interface which
is propelry synchronized like blk_abort_request(), which btw is what
libata already does. Otherwise, it's racy with or without these
patches.

Thanks.

--
tejun