Re: [RFC PATCH v3 02/16] cxl/acpi: Add an acpi_cxl module for the CXL interconnect

From: Dan Williams
Date: Tue Jan 12 2021 - 14:44:07 EST

On Tue, Jan 12, 2021 at 10:44 AM Jonathan Cameron
<Jonathan.Cameron@xxxxxxxxxx> wrote:
>
> On Mon, 11 Jan 2021 14:51:06 -0800
> Ben Widawsky <ben.widawsky@xxxxxxxxx> wrote:
>
> > From: Vishal Verma <vishal.l.verma@xxxxxxxxx>
> >
> > Add an acpi_cxl module to coordinate the ACPI portions of the CXL
> > (Compute eXpress Link) interconnect. This driver binds to ACPI0017
> > objects in the ACPI tree, and coordinates access to the resources
> > provided by the ACPI CEDT (CXL Early Discovery Table).
> >
> > It also coordinates operations of the root port _OSC object to notify
> > platform firmware that the OS has native support for the CXL
> > capabilities of endpoints.
> >
> > Note: the actbl1.h changes are speculative. The expectation is that they
> > will arrive through the ACPICA tree in due time.
>
> I would pull the ACPICA changes out into a precursor patch.


>
> >
> > Cc: Ben Widawsky <ben.widawsky@xxxxxxxxx>
> > Cc: Dan Williams <dan.j.williams@xxxxxxxxx>
> > Signed-off-by: Vishal Verma <vishal.l.verma@xxxxxxxxx>
> > Signed-off-by: Ben Widawsky <ben.widawsky@xxxxxxxxx>
>
> Hi,
>
> I think it would be good to also add CEDT to the list in drivers/acpi/tables.c
> so that we can dump it from /sys/firmware/acpi/tables/ and potentially
> override it from an initrd.

ACPICA changes will eventually come through the ACPI tree not this patch set.


>
> https://elixir.bootlin.com/linux/v5.11-rc3/source/drivers/acpi/tables.c#L482
> Can be very helpful whilst debugging. Related to that, anyone know if anyone
> has acpica patches so we can have iasl -d work on the table? Would probably
> be useful but I'd rather not duplicate work if it's already done.
>

The supplemental tables described here:

https://www.uefi.org/acpi

...do eventually make there way into ACPICA. Added Bob and Erik in
case they can comment on when CEDT and CDAT support will be picked up.

> A few minor things inline
>
> Jonathan
>
> > ---
> > drivers/Kconfig | 1 +
> > drivers/Makefile | 1 +
> > drivers/cxl/Kconfig | 36 ++++++++++++++++
> > drivers/cxl/Makefile | 5 +++
> > drivers/cxl/acpi.c | 97 +++++++++++++++++++++++++++++++++++++++++++
> > drivers/cxl/acpi.h | 15 +++++++
> > include/acpi/actbl1.h | 50 ++++++++++++++++++++++
> > 7 files changed, 205 insertions(+)
> > create mode 100644 drivers/cxl/Kconfig
> > create mode 100644 drivers/cxl/Makefile
> > create mode 100644 drivers/cxl/acpi.c
> > create mode 100644 drivers/cxl/acpi.h
> >
> > diff --git a/drivers/Kconfig b/drivers/Kconfig
> > index dcecc9f6e33f..62c753a73651 100644
> > --- a/drivers/Kconfig
> > +++ b/drivers/Kconfig
> > @@ -6,6 +6,7 @@ menu "Device Drivers"
> > source "drivers/amba/Kconfig"
> > source "drivers/eisa/Kconfig"
> > source "drivers/pci/Kconfig"
> > +source "drivers/cxl/Kconfig"
> > source "drivers/pcmcia/Kconfig"
> > source "drivers/rapidio/Kconfig"
> >
> > diff --git a/drivers/Makefile b/drivers/Makefile
> > index fd11b9ac4cc3..678ea810410f 100644
> > --- a/drivers/Makefile
> > +++ b/drivers/Makefile
> > @@ -73,6 +73,7 @@ obj-$(CONFIG_NVM) += lightnvm/
> > obj-y += base/ block/ misc/ mfd/ nfc/
> > obj-$(CONFIG_LIBNVDIMM) += nvdimm/
> > obj-$(CONFIG_DAX) += dax/
> > +obj-$(CONFIG_CXL_BUS) += cxl/
> > obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf/
> > obj-$(CONFIG_NUBUS) += nubus/
> > obj-y += macintosh/
> > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
> > new file mode 100644
> > index 000000000000..68da926ba5b1
> > --- /dev/null
> > +++ b/drivers/cxl/Kconfig
> > @@ -0,0 +1,36 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +menuconfig CXL_BUS
> > + tristate "CXL (Compute Express Link) Devices Support"
> > + help
> > + CXL is a bus that is electrically compatible with PCI-E, but layers
>
> For consistency with Kconfig in driver/pci probably better to spell PCI Express
> throughout.

Sure.

>
> > + three protocols on that signalling (CXL.io, CXL.cache, and CXL.mem).
> > + The CXL.cache protocol allows devices to hold cachelines locally, the
> > + CXL.mem protocol allows devices to be fully coherent memory targets,
> > + the CXL.io protocol is equivalent to PCI-E. Say 'y' to enable support
> > + for the configuration and management of devices supporting these
> > + protocols.
> > +
> > +if CXL_BUS
> > +
> > +config CXL_BUS_PROVIDER
> > + tristate
> > +
> > +config CXL_ACPI
> > + tristate "CXL ACPI: Platform Support"
> > + depends on ACPI
> > + default CXL_BUS
> > + select CXL_BUS_PROVIDER
> > + help
> > + Say 'y/m' to enable a driver (named "cxl_acpi.ko" when built
> > + as a module) that will enable support for CXL.mem endpoint
> > + devices. In general, CXL Platform Support is a prerequisite
> > + for any CXL device driver that wants to claim ownership of a
> > + component register space. By default platform firmware assumes
> > + Linux is unaware of CXL capabilities and requires explicit
> > + opt-in. This platform component also mediates resources
> > + described by the CEDT (CXL Early Discovery Table). See
> > + Chapter 9.14.1 CXL Early Discovery Table (CEDT) in the CXL 2.0
> > + specification.
> > +
> > + If unsure say 'm'
> > +endif
> > diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile
> > new file mode 100644
> > index 000000000000..d38cd34a2582
> > --- /dev/null
> > +++ b/drivers/cxl/Makefile
> > @@ -0,0 +1,5 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o
> > +
> > +ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=CXL
> > +cxl_acpi-y := acpi.o
> > diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
> > new file mode 100644
> > index 000000000000..0f1ba9b3f1ed
> > --- /dev/null
> > +++ b/drivers/cxl/acpi.c
> > @@ -0,0 +1,97 @@
> > +// SPDX-License-Identifier: GPL-2.0-only
> > +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> > +#include <linux/platform_device.h>
> > +#include <linux/list_sort.h>
> > +#include <linux/module.h>
> > +#include <linux/mutex.h>
> > +#include <linux/sysfs.h>
> > +#include <linux/list.h>
> > +#include <linux/acpi.h>
> > +#include <linux/sort.h>
> > +#include <linux/pci.h>
>
> Trivial: For includes I'm fairly sure general convention (as much as there
> is one is to do alphabetical order, not reverse xmas tree).

No, there's no guidance for this in coding-style so it's one of those
maintainer preference items.

>
> > +#include "acpi.h"
> > +
> > +/*
> > + * If/when CXL support is defined by other platform firmware the kernel
> > + * will need a mechanism to select between the platform specific version
> > + * of this routine, until then, hard-code ACPI assumptions
> > + */
> > +int cxl_bus_acquire(struct pci_dev *pdev)
> > +{
> > + struct acpi_device *adev;
> > + struct pci_dev *root_port;
> > + struct device *root;
> > +
> > + root_port = pcie_find_root_port(pdev);
> > + if (!root_port)
> > + return -ENXIO;
> > +
> > + root = root_port->dev.parent;
> > + if (!root)
> > + return -ENXIO;
> > +
> > + adev = ACPI_COMPANION(root);
> > + if (!adev)
> > + return -ENXIO;
> > +
> > + /* TODO: OSC enabling */
> > +
> > + return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(cxl_bus_acquire);
> > +
> > +static void acpi_cedt_put_table(void *table)
> > +{
> > + acpi_put_table(table);
> > +}
> > +
> > +static int cxl_acpi_probe(struct platform_device *pdev)
> > +{
> > + struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
> > + struct cxl_acpi_desc *acpi_desc;
> > + struct device *dev = &adev->dev;
> > + struct acpi_table_header *tbl;
> > + acpi_status status;
> > + acpi_size sz;
> > + int rc;
> > +
> > + status = acpi_get_table(ACPI_SIG_CEDT, 0, &tbl);
> > + if (ACPI_FAILURE(status)) {
> > + dev_err(dev, "failed to find CEDT at startup\n");
> > + return 0;
> > + }
> > +
> > + rc = devm_add_action_or_reset(dev, acpi_cedt_put_table, tbl);
> > + if (rc)
> > + return rc;
> > +
> > + sz = tbl->length;
> > + dev_info(dev, "found CEDT at startup: %lld bytes\n", sz);
>
> Is this useful? At least for my normal acpi boot I'll see CEDT in the list of
> detected ACPI tables early in boot. This provides less info than that
> print already has.

Agree.