Re: [PATCH net-next v3 3/3] net: ti: icssg-prueth: Add support for ICSSG switch firmware

[Andrew Lunn]

On Thu, Mar 28, 2024 at 11:39:33AM +0530, MD Danish Anwar wrote:
> Hi Andrew,
> 
> On 27/03/24 6:05 pm, Andrew Lunn wrote:
> > On Wed, Mar 27, 2024 at 05:10:54PM +0530, MD Danish Anwar wrote:
> >> Add support for ICSSG switch firmware using existing Dual EMAC driver
> >> with switchdev.
> >>
> >> Limitations:
> >> VLAN offloading is limited to 0-256 IDs.
> >> MDB/FDB static entries are limited to 511 entries and different FDBs can
> >> hash to same bucket and thus may not completely offloaded
> >>
> >> Switch mode requires loading of new firmware into ICSSG cores. This
> >> means interfaces have to taken down and then reconfigured to switch
> >> mode.
> > 
> > Patch 0/3 does not say this. It just shows the interfaces being added
> 
> I will modify the cover letter to state that.
> 
> > to the bridge. There should not be any need to down the interfaces.
> > 
> 
> The interfaces needs to be turned down for switching between dual emac
> and switch mode.
> 
> Dual Emac mode runs with ICSSG Dual Emac firmware where as Switch mode
> works with ICSSG Switch firmware. These firmware are running on the
> dedicated PRU RPROC cores (pru0, rtu0, txpru0). When switch mode is
> enabled, these pru cores need to be stopped and then Switch firmware is
> loaded on these cores and then the cores are started again.
> 
> We stop the cores when interfaces are down and start the cores when
> interfaces are up.
> 
> In short, Dual EMAC firmware runs on pru cores, we put down the
> interface, stop pru cores, load switch firmware on the cores, bring the
> interface up and start the pru cores and now Switch mode is enabled.

This is not the Linux model. Try it, add an interface to a software
bridge. It does not care if it is admin up or down.

You need to hide this difference in your driver.

> > I keep asking this, so it would be good to explain it in the commit
> > message. What configuration is preserved over a firmware reload, and
> > what is lost?
> > 
> > Can i add VLAN in duel MAC mode and then swap into the switch firmware
> > and all the VLANs are preserved? Can i add fdb entries to a port in
> > dual MAC mode, and then swap into the swtich firmware and the FDB
> > table is preserved? What about STP port state? What about ... ?
> > 
> 
> When ports are brought up (firmware reload) we do a full cleaning of all
> the shared memories i.e. SMEM (shared RAM). [1]
> 
> Vlan table and FDB table are stored in SMEM so all the configuration
> done to VLAN / FDB tables will be lost.
> 
> We don't clear DRAM. DRAM is used for sending r30 commands [see
> emac_r30_cmd_init()], configure half duplex [see
> icssg_config_half_duplex()] and configure link speed [see
> icssg_config_set_speed()]. r30 commands are used to set port state (stp).
> 
> Now when the interfaces are brought up (firmware reload) r30 command is
> reconfigured as a result any changes done to port state (stp) will be
> lost. But the duplex and speed settings will be preserved.
> 
> To summarize,
> VLAN table / FDB table and port states are lost during a firmware reload.

So you also need to work around this in your driver. I think it is
possible to get the network stack to enumerate the configuration. Take
a look at the Mellanox driver. If i remember it does something like
this, but i don't remember the details.

      Andrew