Re: [PATCH v12 14/25] irqchip/sifive-plic: Convert PLIC driver into a platform driver

[Anup Patel]

On Fri, Feb 16, 2024 at 9:03 PM Thomas Gleixner <tglx@xxxxxxxxxxxxx> wrote:
>
> On Sat, Jan 27 2024 at 21:47, Anup Patel wrote:
> > +     priv->irqdomain = irq_domain_create_linear(dev->fwnode, nr_irqs + 1,
> > +                                                &plic_irqdomain_ops, priv);
> > +     if (WARN_ON(!priv->irqdomain))
> > +             return -ENOMEM;
>
> While some of the stuff is cleaned up by devm, the error handling in
> this code looks pretty fragile as it leaves initialized contexts,
> hardware state, chained handlers etc. around.

Sure, let me try to improve the error handling.

>
> The question is whether the system can actually boot or work at all if
> any of this fails.

On platforms with PLIC, the PLIC only manages wired interrupts
whereas IPIs are provided through SBI (firmware interface) so a
system can actually continue and boot further without PLIC.

In fact, we do have a synthetic platform (namely QEMU spike)
where there is no PLIC instance and Linux boots using SBI based
polling console.

>
> > +
> >       /*
> >        * We can have multiple PLIC instances so setup cpuhp state
> > -      * and register syscore operations only when context handler
> > -      * for current/boot CPU is present.
> > +      * and register syscore operations only after context handlers
> > +      * of all online CPUs are initialized.
> >        */
> > -     handler = this_cpu_ptr(&plic_handlers);
> > -     if (handler->present && !plic_cpuhp_setup_done) {
> > +     cpuhp_setup = true;
> > +     for_each_online_cpu(cpu) {
> > +             handler = per_cpu_ptr(&plic_handlers, cpu);
> > +             if (!handler->present) {
> > +                     cpuhp_setup = false;
> > +                     break;
> > +             }
> > +     }
> > +     if (cpuhp_setup) {
> >               cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
> >                                 "irqchip/sifive/plic:starting",
> >                                 plic_starting_cpu, plic_dying_cpu);
> >               register_syscore_ops(&plic_irq_syscore_ops);
> > -             plic_cpuhp_setup_done = true;
>
> I don't think that removing the setup protection is correct.
>
> Assume you have maxcpus=N on the kernel command line, then the above
> for_each_online_cpu() loop would result in cpuhp_setup == true when the
> instances for the not onlined CPUs are set up, no?

A platform can have multiple PLIC instances where each PLIC
instance targets a subset of HARTs (or CPUs).

Previously (before this patch), we were probing PLIC very early so on
a platform with multiple PLIC instances, we need to ensure that cpuhp
setup is done only after PLIC context associated with boot CPU is
initialized hence the plic_cpuhp_setup_done check.

This patch converts PLIC driver into a platform driver so now PLIC
instances are probed after all available CPUs are brought-up. In this
case, the cpuhp setup must be done only after PLIC context of all
available CPUs are initialized otherwise some of the CPUs crash
in plic_starting_cpu() due to lack of PLIC context initialization.

Regards,
Anup