RE: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock in hard IRQ

[Song Bao Hua (Barry Song)]

> -----Original Message-----
> From: Alex Williamson [mailto:alex.williamson@xxxxxxxxxx]
> Sent: Wednesday, October 14, 2020 1:50 PM
> To: Song Bao Hua (Barry Song) <song.bao.hua@xxxxxxxxxxxxx>
> Cc: tiantao (H) <tiantao6@xxxxxxxxxxxxx>; eric.auger@xxxxxxxxxx;
> cohuck@xxxxxxxxxx; kvm@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx;
> Linuxarm <linuxarm@xxxxxxxxxx>
> Subject: Re: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock in
> hard IRQ
> 
> On Wed, 14 Oct 2020 00:15:13 +0000
> "Song Bao Hua (Barry Song)" <song.bao.hua@xxxxxxxxxxxxx> wrote:
> 
> > > -----Original Message-----
> > > From: Alex Williamson [mailto:alex.williamson@xxxxxxxxxx]
> > > Sent: Wednesday, October 14, 2020 10:32 AM
> > > To: tiantao (H) <tiantao6@xxxxxxxxxxxxx>
> > > Cc: eric.auger@xxxxxxxxxx; cohuck@xxxxxxxxxx; kvm@xxxxxxxxxxxxxxx;
> > > linux-kernel@xxxxxxxxxxxxxxx; Song Bao Hua (Barry Song)
> > > <song.bao.hua@xxxxxxxxxxxxx>; Linuxarm <linuxarm@xxxxxxxxxx>
> > > Subject: Re: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock
> in
> > > hard IRQ
> > >
> > > On Tue, 13 Oct 2020 10:00:58 +0800
> > > Tian Tao <tiantao6@xxxxxxxxxxxxx> wrote:
> > >
> > > > It is redundant to do irqsave and irqrestore in hardIRQ context.
> > >
> > > But this function is also called from non-IRQ context.  Thanks,
> >
> > It seems you mean
> > vfio_platform_set_irqs_ioctl() ->
> > vfio_platform_set_irq_trigger ->
> > handler() ?
> 
> Yes.
> 
> > so, will it be better to move the irqsave out of the
> vfio_automasked_irq_handler()
> > and put it to where the function is called in non-IRQ context?
> >
> > I mean:
> >
> > irqhandler()
> > {
> > 	spin_lock()  //without irqsave
> > 	spin_unlock()
> > }
> >
> > Non-irq context which is calling this handler:
> > irqsave();
> > irqhandler();
> > irqrestore();
> >
> > Anyway, if it is called in IRQ context, it is redundant to do irqsave.
> 
> What's the advantage?  You're saying it's redundant, is it also wrong?

It is not wrong and it doesn't make any malfunction. It just takes a couple of
instruction cycles to do save/restore and irq-disable/enable of cpu, which
is useless in irq context.

So the advantage is that we are going to remove some redundant instruction
cycles. And if the irq handler is called very often, we speed up the system.

> If it's not wrong and only redundant, what's the tangible latency
> difference in maintaining a separate IRQ context handler without the
> irqsave/restore?  Thanks,

For this question, maybe need some benchmark to get answer. If the irqhandler
is not called that often, I agree it might be not worth to maintain two pieces of
code.

> 
> Alex
> 
> > > > Signed-off-by: Tian Tao <tiantao6@xxxxxxxxxxxxx>
> > > > ---
> > > >  drivers/vfio/platform/vfio_platform_irq.c | 5 ++---
> > > >  1 file changed, 2 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/drivers/vfio/platform/vfio_platform_irq.c
> > > b/drivers/vfio/platform/vfio_platform_irq.c
> > > > index c5b09ec..24fd6c5 100644
> > > > --- a/drivers/vfio/platform/vfio_platform_irq.c
> > > > +++ b/drivers/vfio/platform/vfio_platform_irq.c
> > > > @@ -139,10 +139,9 @@ static int vfio_platform_set_irq_unmask(struct
> > > vfio_platform_device *vdev,
> > > >  static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
> > > >  {
> > > >  	struct vfio_platform_irq *irq_ctx = dev_id;
> > > > -	unsigned long flags;
> > > >  	int ret = IRQ_NONE;
> > > >
> > > > -	spin_lock_irqsave(&irq_ctx->lock, flags);
> > > > +	spin_lock(&irq_ctx->lock);
> > > >
> > > >  	if (!irq_ctx->masked) {
> > > >  		ret = IRQ_HANDLED;
> > > > @@ -152,7 +151,7 @@ static irqreturn_t
> vfio_automasked_irq_handler(int
> > > irq, void *dev_id)
> > > >  		irq_ctx->masked = true;
> > > >  	}
> > > >
> > > > -	spin_unlock_irqrestore(&irq_ctx->lock, flags);
> > > > +	spin_unlock(&irq_ctx->lock);
> > > >
> > > >  	if (ret == IRQ_HANDLED)
> > > >  		eventfd_signal(irq_ctx->trigger, 1);
> >

Thanks
Barry