RE: [EXT] [RFC PATCH 0/4] PCI: endpoint: Introduce a virtio-net EP function

From: Frank Li
Date: Tue Feb 07 2023 - 11:02:09 EST


> We project extending this module to support RDMA. The plan is based on
> virtio-rdma[1].
> It extends the virtio-net and we are plan to implement the proposed
> spec based on this patch.
> [1] virtio-rdma
> - proposal:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> ernel.org%2Fall%2F20220511095900.343-1-
> xieyongji%40bytedance.com%2FT%2F&data=05%7C01%7Cfrank.li%40nxp.co
> m%7C0ef2bd62eda945c413be08db08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5
> c301635%7C0%7C0%7C638113625610341574%7CUnknown%7CTWFpbGZsb3d
> 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> 3D%7C3000%7C%7C%7C&sdata=HyhpRTG8MNx%2BtfmWn6x3srmdBjHcZAo
> 2qbxL9USph9o%3D&reserved=0
> - presentation on kvm forum:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fyout
> u.be%2FQrhv6hC_YK4&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62
> eda945c413be08db08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> 7C0%7C638113625610341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> C%7C%7C&sdata=ucOsGR1letTjxf0gKN6uls5y951CXaIspZtLGnASEC8%3D&res
> erved=0
>

Sorry for our outlook client always change link. This previous discussion.
https://lore.kernel.org/imx/d098a631-9930-26d3-48f3-8f95386c8e50@xxxxxx/T/#t

Look like Endpoint maintainer Kishon like endpoint side work as vhost.
Previous Haotian Wang submit similar patches, which just not use eDMA, just use memcpy.
But overall idea is the same.

I think your and haotian's method is more reasonable for PCI-RC EP connection.

Kishon is not active recently. Maybe need Lorenzo Pieralisi and Bjorn helgass's comments
for overall directions.

Frank Li

> Please feel free to comment and suggest.
> > Frank Li
> >
> > >
> > > To realize the function, this patchset has few changes and introduces a
> > > new APIs to PCI EP framework related to virtio. Furthermore, it device
> > > depends on the some patchtes that is discussing. Those depended
> patchset
> > > are following:
> > > - [PATCH 1/2] dmaengine: dw-edma: Fix to change for continuous
> transfer
> > > link:
> > >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62eda945c413be08db
> 08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6381136256
> 10341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d
> VZMaheX3eR1xA2wQtecmT857h2%2BFtUbhDSHXwgvsEY%3D&reserved=0
> > > ernel.org%2Fdmaengine%2F20221223022608.550697-1-
> > >
> mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
> > > 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> > >
> 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> > >
> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > >
> C%7C%7C&sdata=tIn0MHzEvrdxaC4KKTvTRvYXBzQ6MyrFa2GXpa3ePv0%3D&
> > > reserved=0
> > > - [RFC PATCH 0/3] Deal with alignment restriction on EP side
> > > link:
> > >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62eda945c413be08db
> 08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6381136256
> 10341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d
> VZMaheX3eR1xA2wQtecmT857h2%2BFtUbhDSHXwgvsEY%3D&reserved=0
> > > ernel.org%2Flinux-pci%2F20230113090350.1103494-1-
> > >
> mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
> > > 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> > >
> 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> > >
> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > >
> C%7C%7C&sdata=RLpnDiLwfqQd5QMXdiQyPVCkfOj8q2AyVeZOwWHvlsM%3
> > > D&reserved=0
> > > - [RFC PATCH v2 0/7] Introduce a vringh accessor for IO memory
> > > link:
> > >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62eda945c413be08db
> 08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6381136256
> 10341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d
> VZMaheX3eR1xA2wQtecmT857h2%2BFtUbhDSHXwgvsEY%3D&reserved=0
> > > ernel.org%2Fvirtualization%2F20230202090934.549556-1-
> > >
> mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
> > > 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> > >
> 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> > >
> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > >
> C%7C%7C&sdata=6jgY76BMSbvamb%2Fl3Urjt4Gcizeqon%2BZE5nPssc2kDA%
> > > 3D&reserved=0
> > >
> > > About this patchset has 4 patches. The first of two patch is little changes
> > > to virtio. The third patch add APIs to easily access virtio data structure
> > > on PCIe Host side memory. The last one introduce a virtio-net EP device
> > > function. Details are in commit respectively.
> > >
> > > Currently those network devices are testd using ping only. I'll add a
> > > result of performance evaluation using iperf and etc to the future version
> > > of this patchset.
> > >
> > > Shunsuke Mie (4):
> > > virtio_pci: add a definition of queue flag in ISR
> > > virtio_ring: remove const from vring getter
> > > PCI: endpoint: Introduce virtio library for EP functions
> > > PCI: endpoint: function: Add EP function driver to provide virtio net
> > > device
> > >
> > > drivers/pci/endpoint/Kconfig | 7 +
> > > drivers/pci/endpoint/Makefile | 1 +
> > > drivers/pci/endpoint/functions/Kconfig | 12 +
> > > drivers/pci/endpoint/functions/Makefile | 1 +
> > > .../pci/endpoint/functions/pci-epf-vnet-ep.c | 343 ++++++++++
> > > .../pci/endpoint/functions/pci-epf-vnet-rc.c | 635
> ++++++++++++++++++
> > > drivers/pci/endpoint/functions/pci-epf-vnet.c | 387 +++++++++++
> > > drivers/pci/endpoint/functions/pci-epf-vnet.h | 62 ++
> > > drivers/pci/endpoint/pci-epf-virtio.c | 113 ++++
> > > drivers/virtio/virtio_ring.c | 2 +-
> > > include/linux/pci-epf-virtio.h | 25 +
> > > include/linux/virtio.h | 2 +-
> > > include/uapi/linux/virtio_pci.h | 2 +
> > > 13 files changed, 1590 insertions(+), 2 deletions(-)
> > > create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet-ep.c
> > > create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet-rc.c
> > > create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet.c
> > > create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet.h
> > > create mode 100644 drivers/pci/endpoint/pci-epf-virtio.c
> > > create mode 100644 include/linux/pci-epf-virtio.h
> > >
> > > --
> > > 2.25.1
> >
> Best,
> Shunsuke