Re: [PATCH v9 3/7] genirq: Add mechanism to multiplex a single HW IPI

From: Anup Patel
Date: Mon Nov 14 2022 - 03:57:05 EST


On Sun, Sep 4, 2022 at 1:36 PM Hillf Danton <hdanton@xxxxxxxx> wrote:
>
> On 3 Sep 2022 21:43:05 +0530 Anup Patel <apatel@xxxxxxxxxxxxxxxx> wrote:
> >
> > +static void ipi_mux_send_mask(struct irq_data *d, const struct cpumask *mask)
> > +{
> > + u32 ipi_bit = BIT(irqd_to_hwirq(d));
> > + struct cpumask pmask = { 0 };
> > + unsigned long pending;
> > + int cpu;
> > +
> > + for_each_cpu(cpu, mask) {
> > + pending = atomic_fetch_or_release(ipi_bit,
> > + per_cpu_ptr(&ipi_mux_bits, cpu));
> > +
> > + /*
> > + * The atomic_fetch_or_release() above must complete before
> > + * the atomic_read() below to avoid racing ipi_mux_unmask().
> > + */
> > + smp_mb__after_atomic();
> > +
> > + if (!(pending & ipi_bit) &&
> > + (atomic_read(per_cpu_ptr(&ipi_mux_enable, cpu)) & ipi_bit))
> > + cpumask_set_cpu(cpu, &pmask);
> > + }
> > +
> > + /* Trigger the parent IPI */
> > + ipi_mux_ops->ipi_mux_send(ipi_mux_parent_virq, ipi_mux_data, &pmask);
> > +}
>
> Can you shed some light on the reasons why a pending TLB ipi for flushing
> page3 on CPU2 prevents TLB ipi from being sent to CPU2 for flushing page9.

The "!(pending & ipi_bit)" check in the above "if ()" statement seems to be
causing this. I am seeing CPU soft-lockup due to the "!(pending & ipi_bit)"
check on a QEMU RISC-V virt machine with large CPUs.

I will quickly send v11 with the check removed.

Thanks,
Anup