Re: [PATCH v1 2/2] driver core: fw_devlink: Handle missing drivers for optional suppliers

[Saravana Kannan]

On Tue, Feb 2, 2021 at 12:49 AM Geert Uytterhoeven <geert@xxxxxxxxxxxxxx> wrote:
>
> Hi Saravana,
>
> On Mon, Feb 1, 2021 at 9:49 PM Saravana Kannan <saravanak@xxxxxxxxxx> wrote:
> > On Mon, Feb 1, 2021 at 2:32 AM Geert Uytterhoeven <geert@xxxxxxxxxxxxxx> wrote:
> > > On Sat, Jan 30, 2021 at 5:03 AM Saravana Kannan <saravanak@xxxxxxxxxx> wrote:
> > > > After a deferred probe attempt has exhaused all the devices that can be
> > > > bound, any device that remains unbound has one/both of these conditions
> > > > true:
> > > >
> > > > (1) It is waiting on its supplier to bind
> > > > (2) It does not have a matching driver
> > > >
> > > > So, to make fw_devlink=on more forgiving of missing drivers for optional
> > > > suppliers, after we've done a full deferred probe attempt, this patch
> > > > deletes all device links created by fw_devlink where the supplier hasn't
> > > > probed yet and the supplier itself is not waiting on any of its
> > > > suppliers. This allows consumers to probe during another deferred probe
> > > > attempt if they were waiting on optional suppliers.
> > > >
> > > > When modules are enabled, we can't differentiate between a driver
> > > > that'll never be registered vs a driver that'll be registered soon by
> > > > loading a module. So, this patch doesn't do anything for the case where
> > > > modules are enabled.
> > >
> > > For the modular case, can't you do a probe regardless? Or limit it
> > > to devices where the missing provider is a DMAC or IOMMU driver?
> > > Many drivers can handle missing DMAC controller drivers, and are even
> > > supposed to work that way.  They may even retry obtaining DMA releases
> > > later.
> >
> > I don't want to handle this at a property/provider-type level. It'll
> > be a whack-a-mole that'll never end -- there'll be some driver that
> > would work without some resource. Letting it probe is not difficult (I
> > just need to drop these device links), but the problem is that a lot
> > of drivers are not written properly to be able to handle getting
> > deferred and then getting reattempted before the supplier. Either
> > because:
> >
> > 1. They were never built and tested as a module
> > 2. The supplier gets deferred and the consumer doesn't have proper
> > deferred probe implementation and when we drop the device links, the
> > consumer might be attempted before the supplier and things go bad.
>
> You may be a bit too pessimistic here: we had deferred probing for
> years.  With devices with complex dependencies, it's not uncommon for a
> driver to be probed 3 or 4 times, before it succeeds (and FTR, would be
> happy to see this fixed).  So most drivers should handle this already.
> And if they don't, they're already broken.

I fully agree the drivers are broken and they should be fixed (like
we've been doing so far). I'd happily say "fix your driver" and just
mark properties (iommu, dmas) as "optional" and be done with it.
Actually, handling optional properties is pretty simple -- we just
need to stop parsing them (delete code/make it a flag).

But my understanding is, you can't set fw_devlink=on by default and
break devices that used to boot fine before. But we can't find the
broken drivers without setting fw_devlink=on by default. So it's a
catch-22. I'm happy to continue helping debug the issues if we are
okay with leaving fw_devlink=on in 5.12, but I'm not sure everyone
will agree to that.

Also, there's also no way to selectively enable fw_devlink on a
DT/board level (Rob will have to agree to a property in the chosen {}
node). So, without that, I'm forced to go for the lowest common
denominator.

>
> > One hack I'm thinking of is that with CONFIG_MODULES, I can drop these
> > unmet device links after a N-second timeout, but having the timeout
> > extended everytime a new driver is registered. So as long as no two
> > modules are loaded further than N seconds apart during boot up, it
> > would all just work out fine. But it doesn't solve the problem fully
> > either. But maybe it'll be good enough? I haven't analyzed this fully
> > yet -- so apologies in advance if it's stupid.
>
> So you would introduce an additional delay when e.g. mounting the root
> file system, as the SDHI driver probe may be postponed due to the DMAC
> driver not being available?

Honestly I'm thinking of deleting/adding a conditional for iommu/dma
parsing. So boards that know all iommus/dmas are needed can set some
flag if they want.

In your example, if the DMAC driver is needed for SDHI to work (mount
root), then I'd expect it to be loaded(ramdisk)/builtin before mount
root. And it'd just work. If the DMA is optional and SDHI should have
mounted root without it, then the solution is what I said above.

> Let's consider the possible combinations here:
>   1. If both provider and consumer are built-in, there's no issue.
>   2. If the provider is built-in, and the consumer is modular, it should
>      just work, too.
>   3. If the consumer is built-in, but the provider is modular (or not
>      available), what to do?
>      Wouldn't it be safe to assume the user did intend the consumer to
>      probe without the provider, and thus continue, if possible (e.g.
>      for an optional DMA channel)?  Else the user would have configured
>      the provider driver built-in, too.
>   4. If both provider and consumer are modular, perhaps userspace should
>      try to load the modules in the right order, i.e. provider drivers
>      first?

One of the main benefits/goal of fw_devlink is that it makes module
load ordering irrelevant. There are other correctness issues with
depending on module load ordering too. So I don't want to go down the
(fix your module load order) route.

>
> For DMACs, it's the consumer that knows if it can work without the DMAC
> driver present, and fall back to PIO, or not.  So we could add a flag
> for that to struct driver.

The whole point of fw_devlink is to figure stuff out without the
drivers being available (sync_state() is a whole another layer of
complexity). If the drivers come into picture, we don't really need
fw_devlink to parse dmas. The dma framework code can add device links
once a consumer starts using the dma (similar to how iommus do it).

> For IOMMUs, it's different: the consumer drivers are not aware of it,
> only the driver/IOMMU subsystem is.  And whether to work in the absence
> of an IOMMU driver is more like a system (security) policy decision.

I think some consumers can make this decision too. For example, not
supporting DRM content if the IOMMU isn't available, etc.

Long story short, I don't have a problem with saying fix the driver.
But I don't  think I can have fw_devlink=on by default with that
argument. And making it fw_devlink=on by default is my goal.

-Saravana