Re: PHY issue with SJA1105Q/DP84849I Design

From: Jagan Teki
Date: Fri Dec 29 2023 - 15:19:01 EST

On Fri, Dec 29, 2023 at 8:55 PM Vladimir Oltean <olteanv@xxxxxxxxx> wrote:
>
> On Fri, Dec 29, 2023 at 05:12:39PM +0530, Jagan Teki wrote:
> > With fec0 fixed-link and 3 different switch port configurations, the
> > result of the link seems to be up but the ping not working and even
> > the packets are not transmitted via eth0.
> >
> > DT Combinations:
> >
> > - Port0 is ethphy0, Port1 is ethphy1, Port2 is disabled, Port3 is
> > disabled, Port4 is FEC
> > - Port0 is disabled, Port1 is ethphy0, Port2 is ethphy1, Port3 is
> > disabled, Port4 is FEC
> > - Port0 is disabled, Port1 is disabled, Port2 is ethphy0, Port3 is
> > ethphy1, Port4 as FEC
>
> Why all these combinations? You don't know which switch port is which?

This is where I get confused in the first place. I didn't find proper
information on binding about how the physical pin-out is to be
configured in DT ports or maybe I didn't understand properly.

As per schematics.
Pin MII0_RXD0-D3/TXD0-D3 is connected to PROC_MII0_RXD0-D3/TXD0-D3
this would be fec0.
Pin MII1_RXD0-D3/TXD0-D3 is connected to PHY_MII1_RXD0-D3/TXD0-D3 this
would be ethphy0.
Pin MII2_RXD0-D3/TXD0-D3 is connected to PHY_MII2_RXD0-D3/TXD0-D3 this
would be ethphy1.
Pin MII3_RXD0-D3/TXD0-D3 is grounded
Pin MII4_RXD0-D3/TXD0-D3 is grounded

So, I did use the above 3 combinations and assumed fec0 is always a
port4 based on existing DTS in the tree. Please let me know which
configuration is proper as per schematic connections.

>
> > DT: (with Port0 is ethphy0, Port1 is ethphy1, Port2 is disabled, Port3
> > is disabled, Port4 is FEC)
> >
> > &ecspi2 {
> > cs-gpios = <&gpio2 27 GPIO_ACTIVE_HIGH>;
> > pinctrl-names = "default";
> > pinctrl-0 = <&pinctrl_ecspi2>;
> > status = "okay";
> >
> > switch@0 {
> > compatible = "nxp,sja1105q";
> > reg = <0>;
> > spi-max-frequency = <4000000>;
> > spi-rx-delay-us = <1>;
> > spi-tx-delay-us = <1>;
> > spi-cpha;
> >
> > clocks = <&clk25m>;
> >
> > pinctrl-0 = <&pinctrl_sja1105_rst>;
> > pinctrl-names = "default";
> > reset-gpios = <&gpio6 5 GPIO_ACTIVE_LOW>;
> >
> > ports {
> > #address-cells = <1>;
> > #size-cells = <0>;
> >
> > port@0 {
> > reg = <0>;
> > label = "ethphy0";
> > phy-handle = <&ethphy0>;
> > phy-mode = "mii";
> > };
> >
> > port@1 {
> > reg = <1>;
> > label = "ethphy1";
> > phy-handle = <&ethphy1>;
> > phy-mode = "mii";
> > };
> >
> > port@2 {
> > reg = <2>;
> > status = "disabled";
> > };
> >
> > port@3 {
> > reg = <3>;
> > status = "disabled";
> > };
> >
> > port@4 {
> > reg = <4>;
> > label = "cpu";
> > ethernet = <&fec>;
> > phy-mode = "mii";
> > rx-internal-delay-ps = <2000>;
> > tx-internal-delay-ps = <2000>;
>
> This looks suspicious. "rx-internal-delay-ps" and "tx-internal-delay-ps"
> are only relevant for the RGMII modes, but you specify phy-mode = "mii".
> Does the board schematic confirm that MII is the physical connection
> being used from the switch to the FEC?

Yes, the design uses MII.

>
> If you are truly using MII, then you should remove the RGMII delay
> properties, and since you are using a 6.1 kernel - hence after kernel
> commit 5d645df99ac6 ("net: dsa: sja1105: determine PHY/MAC role from PHY
> interface type") - you should be using phy-mode = "rev-mii" to put this
> port in MII PHY ("RevMII") mode - to interoperate with the FEC in MII
> MAC mode.

Okay, I will remove RGMII delay. is phy-mode = "rev-mii" applicable to
all ports or only for fec port4?

>
> >
> > fixed-link {
> > speed = <100>;
> > full-duplex;
> > };
> > };
> > };
> > };
> > };
> >
> > &fec {
> > pinctrl-names = "default";
> > pinctrl-0 = <&pinctrl_enet>;
> > phy-mode = "mii";
> > status = "okay";
> >
> > fixed-link {
> > speed = <100>;
> > full-duplex;
> > };
> >
> > mdio {
> > #address-cells = <1>;
> > #size-cells = <0>;
> >
> > ethphy0: ethernet-phy@0 {
> > compatible = "ethernet-phy-ieee802.3-c22";
> > reg = <0>;
> > };
> >
> > ethphy1: ethernet-phy@1 {
> > compatible = "ethernet-phy-ieee802.3-c22";
> > reg = <1>;
> > };
> > };
> > };
> >
> > root@ltts-imx6solo:~# bash /usr/phynew.sh
> > ======= MDIO: PHY0 ========
> > [ 162.426515] mdio_netlink: loading out-of-tree module taints kernel.
>
> Still, please refrain from involving out-of-tree modules when asking for
> help upstream. Thanks.

Sorry, I didn't remove it during this log, but I did remove this
module at runtime during my actual sanity tests.

>
> > root@ltts-imx6solo:~# [ 165.208656] sja1105 spi1.0 ethphy0: Link is
> > Up - 100Mbps/Full - flow control off
> > [ 165.225788] sja1105 spi1.0 ethphy1: Link is Up - 100Mbps/Full -
> > flow control off
> > [ 165.235925] IPv6: ADDRCONF(NETDEV_CHANGE): ethphy0: link becomes ready
> > [ 165.255777] IPv6: ADDRCONF(NETDEV_CHANGE): ethphy1: link becomes ready
> >
> > root@ltts-imx6solo:~# ifconfig
> > eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1504
> > inet6 fe80::68fb:8ff:fedf:d377 prefixlen 64 scopeid 0x20<link>
> > ether 6a:fb:08:df:d3:77 txqueuelen 1000 (Ethernet)
> > RX packets 0 bytes 0 (0.0 B)
> > RX errors 0 dropped 0 overruns 0 frame 0
> > TX packets 0 bytes 0 (0.0 B)
> > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
> >
> > ethphy0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> > inet 169.254.178.1 netmask 255.255.0.0 broadcast 0.0.0.0
> > inet6 fe80::211:22ff:fe33:4455 prefixlen 64 scopeid 0x20<link>
> > ether 00:11:22:33:44:55 txqueuelen 1000 (Ethernet)
> > RX packets 0 bytes 0 (0.0 B)
> > RX errors 0 dropped 0 overruns 0 frame 0
> > TX packets 30 bytes 4071 (3.9 KiB)
> > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
> >
> > ethphy1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> > inet 169.253.178.2 netmask 255.255.0.0 broadcast 0.0.0.0
> > inet6 fe80::211:22ff:fe33:4466 prefixlen 64 scopeid 0x20<link>
> > ether 00:11:22:33:44:66 txqueuelen 1000 (Ethernet)
> > RX packets 0 bytes 0 (0.0 B)
> > RX errors 0 dropped 0 overruns 0 frame 0
> > TX packets 30 bytes 4071 (3.9 KiB)
> > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
> >
> > lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
> > inet 127.0.0.1 netmask 255.0.0.0
> > inet6 ::1 prefixlen 128 scopeid 0x10<host>
> > loop txqueuelen 1000 (Local Loopback)
> > RX packets 89 bytes 7675 (7.4 KiB)
> > RX errors 0 dropped 0 overruns 0 frame 0
> > TX packets 89 bytes 7675 (7.4 KiB)
> > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> ifconfig reports statistics counters from the /proc/net/dev interface,
> which the sja1105 does not report very well (they don't come from hardware).
> It's best to use "ethtool -S eth0 | grep -v ': 0'" for FEC and SJA1105
> CPU port (named "p04_*") counters, and "ethtool -S ethphy0 | grep -v ': 0'"
> to get hardware counters from the switch user ports.

Okay.

>
> You can also use the RX counters to determine which switch port is which
> (but the phy-handle of each port to each PHY needs to be correct).

RX counters in ethtool you mean?

>
> > mytsl02383@MYTSL02383:~$ ifconfig -a
> > enp43s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> > inet 169.254.178.2 netmask 255.255.0.0 broadcast 169.254.255.255
> > inet6 fe80::d71b:4bdd:27bd:2a1a prefixlen 64 scopeid 0x20<link>
> > ether 00:be:43:20:9a:26 txqueuelen 1000 (Ethernet)
> > RX packets 272356 bytes 27099064 (27.0 MB)
> > RX errors 0 dropped 19 overruns 0 frame 0
> > TX packets 862 bytes 300806 (300.8 KB)
> > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
> > mytsl02383@MYTSL02383:~$ ping 169.254.178.1
> > PING 169.254.178.1 (169.254.178.1) 56(84) bytes of data.
> > From 169.254.178.2 icmp_seq=1 Destination Host Unreachable
> >
> > Let me know if you need any more details.
>
> I'm not convinced packets are routed through ethphy0 or ethphy1, since
> all interfaces have IPv4 link-local addresses only. You can use
> "ip route get 169.254.178.1" to confirm what interface gets chosen.
> This is not indicative of a device-level problem, just a setup one.
> Please set up some IPv4 static addresses which are not link-local on the
> DSA user ports and try to ping a link partner which has an IP address in
> the same subnet.

Okay. I will update this test along with DT changes.

Thanks,
Jagan.