Re: [V3] PCI: fu740: Drop to 2.5GT/s to fix initial device probing on some boards

[Alexandre Ghiti]

On 3/21/22 14:49, Alexandre Ghiti wrote:
> Hi Ben,
>
> On 3/18/22 16:24, Ben Dooks wrote:
> > The fu740 PCIe core does not probe any devices on the SiFive Unmatched
> > board without this fix (or having U-Boot explicitly start the PCIe via
> > either boot-script or user command). The fix is to start the link at
> > 2.5GT/s speeds and once the link is up then change the maximum speed 
> > back
> > to the default.
> >
> > The U-Boot driver claims to set the link-speed to 2.5GT/s to get the 
> > probe
> > to work (and U-Boot does print link up at 2.5GT/s) in the following 
> > code:
> > https://source.denx.de/u-boot/u-boot/-/blob/master/drivers/pci/pcie_dw_sifive.c?id=v2022.01#L271 
> >
> >
> > Signed-off-by: Ben Dooks <ben.dooks@xxxxxxxxxxxxxxx>
> > --
> > Note, this patch has had significant re-work since the previous 4
> > sets, including trying to fix style, message, reliance on the U-Boot
> > fix and the comments about usage of LINK_CAP and reserved fields.
> >
> > v2:
> > - fix issues with Gen1/2.5GTs
> > - updated comment on the initial probe
> > - run tests with both uninitialised and initialsed pcie from uboot
> > ---
> >   drivers/pci/controller/dwc/pcie-fu740.c | 52 ++++++++++++++++++++++++-
> >   1 file changed, 51 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/pci/controller/dwc/pcie-fu740.c 
> > b/drivers/pci/controller/dwc/pcie-fu740.c
> > index 842b7202b96e..ecac0364178a 100644
> > --- a/drivers/pci/controller/dwc/pcie-fu740.c
> > +++ b/drivers/pci/controller/dwc/pcie-fu740.c
> > @@ -181,10 +181,60 @@ static int fu740_pcie_start_link(struct dw_pcie 
> > *pci)
> >   {
> >       struct device *dev = pci->dev;
> >       struct fu740_pcie *afp = dev_get_drvdata(dev);
> > +    u8 cap_exp = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP);
> > +    int ret;
> > +    u32 orig, tmp;
> > +
> > +    /*
> > +     * Force 2.5GT/s when starting the link, due to some devices not
> > +     * probing at higher speeds. This happens with the PCIe switch
> > +     * on the Unmatched board when U-Boot has not initialised the PCIe.
> > +     * The fix in U-Boot is to force 2.5GT/s, which then gets cleared
> > +     * by the soft reset does by this driver.
> > +     */
> > +
> > +    dev_dbg(dev, "cap_exp at %x\n", cap_exp);
> > +    dw_pcie_dbi_ro_wr_en(pci);
> > +
> > +    tmp = dw_pcie_readl_dbi(pci, cap_exp + PCI_EXP_LNKCAP);
> > +    orig = tmp & PCI_EXP_LNKCAP_SLS;
> > +    tmp &= ~PCI_EXP_LNKCAP_SLS;
> > +    tmp |= PCI_EXP_LNKCAP_SLS_2_5GB;
> > +    dw_pcie_writel_dbi(pci, cap_exp + PCI_EXP_LNKCAP, tmp);
> >         /* Enable LTSSM */
> >       writel_relaxed(0x1, afp->mgmt_base + PCIEX8MGMT_APP_LTSSM_ENABLE);
> > -    return 0;
> > +
> > +    ret = dw_pcie_wait_for_link(pci);
> > +    if (ret) {
> > +        dev_err(dev, "error: link did not start\n");
> > +        goto err;
> > +    }
> > +
> > +    tmp = dw_pcie_readl_dbi(pci, cap_exp + PCI_EXP_LNKCAP);
> > +    if ((tmp & PCI_EXP_LNKCAP_SLS) != orig) {
> > +        dev_dbg(dev, "changing speed back to original\n");
> > +
> > +        tmp &= ~PCI_EXP_LNKCAP_SLS;
> > +        tmp |= orig;
> > +        dw_pcie_writel_dbi(pci, cap_exp + PCI_EXP_LNKCAP, tmp);
> > +
> > +        tmp = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
> > +        tmp |= PORT_LOGIC_SPEED_CHANGE;
> > +        dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, tmp);
> > +
> > +        ret = dw_pcie_wait_for_link(pci);
> > +        if (ret) {
> > +            dev_err(dev, "error: link did not start at new speed\n");
> > +            goto err;
> > +        }
> > +    }
> > +
> > +    ret = 0;
> > +err:
> > +    WARN_ON(ret);    /* we assume that errors will be very rare */
> > +    dw_pcie_dbi_ro_wr_dis(pci);
> > +    return ret;
> >   }
> >     static int fu740_pcie_host_init(struct pcie_port *pp)
>
> +cc Maciej and David as there is this other fix that seems to do the 
> same but differently, it's been under review for some time now: 
> https://lore.kernel.org/all/20220302000043.GA662523@bhelgaas/t/
>
> I fell onto this issue recently, I'll give your patch and the above a 
> try soon.


FWIW, I have tested this and it solved my issue with nvme not being 
probed, so:

Tested-by: Alexandre Ghiti <alexandre.ghiti@xxxxxxxxxxxxx>

Thanks,

Alex


> Thanks
>
> Alex
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@xxxxxxxxxxxxxxxxxxx
> http://lists.infradead.org/mailman/listinfo/linux-riscv