Re: [PATCH 0/9] vfio/fsl-mc: VFIO support for FSL-MC devices

[Diana Craciun OSS]

Thanks for looking into this.

On 3/27/2020 11:11 PM, Alex Williamson wrote:
> On Mon, 23 Mar 2020 19:19:02 +0200
> Diana Craciun <diana.craciun@xxxxxxxxxxx> wrote:
>
> > DPAA2 (Data Path Acceleration Architecture) consists in
> > mechanisms for processing Ethernet packets, queue management,
> > accelerators, etc.
> >
> > The Management Complex (mc) is a hardware entity that manages the DPAA2
> > hardware resources. It provides an object-based abstraction for software
> > drivers to use the DPAA2 hardware. The MC mediates operations such as
> > create, discover, destroy of DPAA2 objects.
> > The MC provides memory-mapped I/O command interfaces (MC portals) which
> > DPAA2 software drivers use to operate on DPAA2 objects.
> >
> > A DPRC is a container object that holds other types of DPAA2 objects.
> > Each object in the DPRC is a Linux device and bound to a driver.
> > The MC-bus driver is a platform driver (different from PCI or platform
> > bus). The DPRC driver does runtime management of a bus instance. It
> > performs the initial scan of the DPRC and handles changes in the DPRC
> > configuration (adding/removing objects).
> >
> > All objects inside a container share the same hardware isolation
> > context, meaning that only an entire DPRC can be assigned to
> > a virtual machine.
> > When a container is assigned to a virtual machine, all the objects
> > within that container are assigned to that virtual machine.
> > The DPRC container assigned to the virtual machine is not allowed
> > to change contents (add/remove objects) by the guest. The restriction
> > is set by the host and enforced by the mc hardware.
> >
> > The DPAA2 objects can be directly assigned to the guest. However
> > the MC portals (the memory mapped command interface to the MC) need
> > to be emulated because there are commands that configure the
> > interrupts and the isolation IDs which are virtual in the guest.
> >
> > Example:
> > echo vfio-fsl-mc > /sys/bus/fsl-mc/devices/dprc.2/driver_override
> > echo dprc.2 > /sys/bus/fsl-mc/drivers/vfio-fsl-mc/bind
> >
> > The dprc.2 is bound to the VFIO driver and all the objects within
> > dprc.2 are going to be bound to the VFIO driver.
> What's the composition of the IOMMU group, does it start with the DPRC
> and each of the objects within the container are added to the same
> group as they're created?

Yes, the IOMMU group starts with the DPRC and the other objects are then 
added to that group.

> For an alternative to the driver_override mechanism used in this series
> of passing the override through various scan/create callbacks, you
> might consider something like I did for PCI SR-IOV:
>
> https://lore.kernel.org/lkml/158396395214.5601.11207416598267070486.stgit@xxxxxxxxxx/
>
> ie. using the bus notifier to setup the driver_override before driver
> matching is done.  Thanks,

Thanks, I like your approach. I will give it a try.

Diana


> Alex
>
> > More details about the DPAA2 objects can be found here:
> > Documentation/networking/device_drivers/freescale/dpaa2/overview.rst
> >
> > The patches are dependent on some changes in the mc-bus (bus/fsl-mc)
> > driver. The changes were needed in order to re-use code and to export
> > some more functions that are needed by the VFIO driver.
> > Currenlty the mc-bus patches are under review:
> > https://www.spinics.net/lists/kernel/msg3447567.html
> >
> > Bharat Bhushan (1):
> >    vfio/fsl-mc: Add VFIO framework skeleton for fsl-mc devices
> >
> > Diana Craciun (8):
> >    vfio/fsl-mc: Scan DPRC objects on vfio-fsl-mc driver bind
> >    vfio/fsl-mc: Implement VFIO_DEVICE_GET_INFO ioctl
> >    vfio/fsl-mc: Implement VFIO_DEVICE_GET_REGION_INFO ioctl call
> >    vfio/fsl-mc: Allow userspace to MMAP fsl-mc device MMIO regions
> >    vfio/fsl-mc: Added lock support in preparation for interrupt handling
> >    vfio/fsl-mc: Add irq infrastructure for fsl-mc devices
> >    vfio/fsl-mc: trigger an interrupt via eventfd
> >    vfio/fsl-mc: Add read/write support for fsl-mc devices
> >
> >   MAINTAINERS                               |   6 +
> >   drivers/vfio/Kconfig                      |   1 +
> >   drivers/vfio/Makefile                     |   1 +
> >   drivers/vfio/fsl-mc/Kconfig               |   9 +
> >   drivers/vfio/fsl-mc/Makefile              |   4 +
> >   drivers/vfio/fsl-mc/vfio_fsl_mc.c         | 660 ++++++++++++++++++++++
> >   drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c    | 221 ++++++++
> >   drivers/vfio/fsl-mc/vfio_fsl_mc_private.h |  56 ++
> >   include/uapi/linux/vfio.h                 |   1 +
> >   9 files changed, 959 insertions(+)
> >   create mode 100644 drivers/vfio/fsl-mc/Kconfig
> >   create mode 100644 drivers/vfio/fsl-mc/Makefile
> >   create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc.c
> >   create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c
> >   create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc_private.h