RE: [PATCH net-next v2] net: macb: In shared MDIO usecase make MDIO producer ethernet node to probe first

[Pandey, Radhey Shyam]

> -----Original Message-----
> From: Saravana Kannan <saravanak@xxxxxxxxxx>
> Sent: Saturday, July 16, 2022 12:39 AM
> To: Pandey, Radhey Shyam <radhey.shyam.pandey@xxxxxxx>
> Cc: Andrew Lunn <andrew@xxxxxxx>; nicolas.ferre@xxxxxxxxxxxxx;
> claudiu.beznea@xxxxxxxxxxxxx; davem@xxxxxxxxxxxxx;
> edumazet@xxxxxxxxxx; kuba@xxxxxxxxxx; pabeni@xxxxxxxxxx;
> hkallweit1@xxxxxxxxx; linux@xxxxxxxxxxxxxxx;
> gregkh@xxxxxxxxxxxxxxxxxxx; rafael@xxxxxxxxxx; netdev@xxxxxxxxxxxxxxx;
> linux-kernel@xxxxxxxxxxxxxxx; git (AMD-Xilinx) <git@xxxxxxx>
> Subject: Re: [PATCH net-next v2] net: macb: In shared MDIO usecase make
> MDIO producer ethernet node to probe first
> 
> On Fri, Jul 15, 2022 at 12:00 PM Pandey, Radhey Shyam
> <radhey.shyam.pandey@xxxxxxx> wrote:
> >
> > > -----Original Message-----
> > > From: Saravana Kannan <saravanak@xxxxxxxxxx>
> > > Sent: Wednesday, July 6, 2022 12:28 AM
> > > To: Pandey, Radhey Shyam <radhey.shyam.pandey@xxxxxxx>
> > > Cc: Andrew Lunn <andrew@xxxxxxx>; nicolas.ferre@xxxxxxxxxxxxx;
> > > claudiu.beznea@xxxxxxxxxxxxx; davem@xxxxxxxxxxxxx;
> > > edumazet@xxxxxxxxxx; kuba@xxxxxxxxxx; pabeni@xxxxxxxxxx;
> > > hkallweit1@xxxxxxxxx; linux@xxxxxxxxxxxxxxx;
> > > gregkh@xxxxxxxxxxxxxxxxxxx; rafael@xxxxxxxxxx;
> > > netdev@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; git
> > > (AMD-Xilinx) <git@xxxxxxx>
> > > Subject: Re: [PATCH net-next v2] net: macb: In shared MDIO usecase
> > > make MDIO producer ethernet node to probe first
> > >
> > > On Tue, Jul 5, 2022 at 11:49 AM Pandey, Radhey Shyam
> > > <radhey.shyam.pandey@xxxxxxx> wrote:
> > > >
> > > > > -----Original Message-----
> > > > > From: Andrew Lunn <andrew@xxxxxxx>
> > > > > Sent: Friday, July 1, 2022 2:44 PM
> > > > > To: Pandey, Radhey Shyam <radhey.shyam.pandey@xxxxxxx>
> > > > > Cc: nicolas.ferre@xxxxxxxxxxxxx; claudiu.beznea@xxxxxxxxxxxxx;
> > > > > davem@xxxxxxxxxxxxx; edumazet@xxxxxxxxxx; kuba@xxxxxxxxxx;
> > > > > pabeni@xxxxxxxxxx; hkallweit1@xxxxxxxxx; linux@xxxxxxxxxxxxxxx;
> > > > > gregkh@xxxxxxxxxxxxxxxxxxx; rafael@xxxxxxxxxx;
> > > saravanak@xxxxxxxxxx;
> > > > > netdev@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; git
> > > > > (AMD-Xilinx) <git@xxxxxxx>
> > > > > Subject: Re: [PATCH net-next v2] net: macb: In shared MDIO
> > > > > usecase make MDIO producer ethernet node to probe first
> > > > >
> > > > > On Fri, Jul 01, 2022 at 01:25:06AM +0530, Radhey Shyam Pandey
> wrote:
> > > > > > In shared MDIO suspend/resume usecase for ex. with MDIO
> > > > > > producer
> > > > > > (0xff0c0000) eth1 and MDIO consumer(0xff0b0000) eth0 there is
> > > > > > a constraint that ethernet interface(ff0c0000) MDIO bus
> > > > > > producer has to be resumed before the consumer ethernet
> interface(ff0b0000).
> > > > > >
> > > > > > However above constraint is not met when GEM0(ff0b0000) is
> > > > > > resumed
> > > first.
> > > > > > There is phy_error on GEM0 and interface becomes
> > > > > > non-functional on
> > > > > resume.
> > > > > >
> > > > > > suspend:
> > > > > > [ 46.477795] macb ff0c0000.ethernet eth1: Link is Down [
> > > > > > 46.483058] macb ff0c0000.ethernet: gem-ptp-timer ptp clock
> > > unregistered.
> > > > > > [ 46.490097] macb ff0b0000.ethernet eth0: Link is Down [
> > > > > > 46.495298] macb ff0b0000.ethernet: gem-ptp-timer ptp clock
> > > unregistered.
> > > > > >
> > > > > > resume:
> > > > > > [ 46.633840] macb ff0b0000.ethernet eth0: configuring for
> > > > > > phy/sgmii link mode macb_mdio_read ->
> > > > > > pm_runtime_get_sync(GEM1)
> > > it
> > > > > > return -
> > > > > EACCES error.
> > > > > >
> > > > > > The suspend/resume is dependent on probe order so to fix this
> > > > > > dependency ensure that MDIO producer ethernet node is always
> > > > > > probed first followed by MDIO consumer ethernet node.
> > > > > >
> > > > > > During MDIO registration find out if MDIO bus is shared and
> > > > > > check if MDIO producer platform node(traverse by 'phy-handle'
> > > > > > property) is bound. If not bound then defer the MDIO consumer
> > > > > > ethernet node
> > > probe.
> > > > > > Doing it ensures that in suspend/resume MDIO producer is
> > > > > > resumed followed by MDIO consumer ethernet node.
> > > > >
> > > > > I don't think there is anything specific to MACB here. There are
> > > > > Freescale boards which have an MDIO bus shared by two interfaces
> etc.
> > > > >
> > > > > Please try to solve this in a generic way, not specific to one
> > > > > MAC and MDIO combination.
> > > >
> > > > Thanks for the review.  I want to get your thoughts on the outline
> > > > of the generic solution. Is the current approach fine and we can
> > > > extend it for all shared MDIO use cases/ or do we see any limitations?
> > > >
> > > > a) Figure out if the MDIO bus is shared.  (new binding or reuse
> > > > existing)
> > > > b) If the MDIO bus is shared based on DT property then figure out
> > > > if the MDIO producer platform device is probed. If not, defer MDIO
> > > > consumer MDIO bus registration.
> > >
> > > Radhey,
> > >
> > > I think Andrew added me because he's pointing you towards fw_devlink.
> > >
> > > Andrew,
> > >
> > > I have intentionally not added phy-handle support to fw_devlink
> > > because it would also prevent the generic driver from binding/cause
> > > issues with DSA. I have some high level ideas on fixing that but haven't
> gotten around to it yet.
> >
> > Thanks, just want to understand on implementation when phy-handle
> > support is added to fw_devlink. Does it ensure that supplier node is
> > probed first? Or it uses device_link framework to specify
> > suspend/resume dependency and don't care on consumer/producer probe
> order.
> 
> fw_devlink will enforce probe ordering and suspend/resume ordering.
> Btw, fw_devlink uses device links underneath. It just used the firmware (Eg:
> DT) to figure out the dependencies. That's why it's called fw_devlink.
Thanks! Forgot to ask earlier, when are you planning to add phy-handle support 
to fw_devlink ? seems like we have a dependency on this feature to make
shared MDIO use case work in a generic way.