Re: [EXT] [RFC PATCH 0/4] PCI: endpoint: Introduce a virtio-net EP function

[Shunsuke Mie]

2023年2月4日(土) 1:45 Frank Li <frank.li@xxxxxxx>:
>
>
>
> > -----Original Message-----
> > From: Shunsuke Mie <mie@xxxxxxxxxx>
> > Sent: Friday, February 3, 2023 4:04 AM
> > To: Lorenzo Pieralisi <lpieralisi@xxxxxxxxxx>
> > Cc: Krzysztof Wilczyński <kw@xxxxxxxxx>; Manivannan Sadhasivam
> > <mani@xxxxxxxxxx>; Kishon Vijay Abraham I <kishon@xxxxxxxxxx>; Bjorn
> > Helgaas <bhelgaas@xxxxxxxxxx>; Michael S. Tsirkin <mst@xxxxxxxxxx>;
> > Jason Wang <jasowang@xxxxxxxxxx>; Shunsuke Mie <mie@xxxxxxxxxx>;
> > Frank Li <frank.li@xxxxxxx>; Jon Mason <jdmason@xxxxxxxx>; Ren Zhijie
> > <renzhijie2@xxxxxxxxxx>; Takanari Hayama <taki@xxxxxxxxxx>; linux-
> > kernel@xxxxxxxxxxxxxxx; linux-pci@xxxxxxxxxxxxxxx; virtualization@lists.linux-
> > foundation.org
> > Subject: [EXT] [RFC PATCH 0/4] PCI: endpoint: Introduce a virtio-net EP
> > function
> >
> > Caution: EXT Email
> >
> > This patchset introduce a virtio-net EP device function. It provides a
> > new option to communiate between PCIe host and endpoint over IP.
> > Advantage of this option is that the driver fully uses a PCIe embedded DMA.
> > It is used to transport data between virtio ring directly each other. It
> > can be expected to better throughput.
>
> Thanks, basic that's what I want.  I am trying use RDMA.
> But I think virtio-net still be good solution.
We project extending this module to support RDMA. The plan is based on
virtio-rdma[1].
It extends the virtio-net and we are plan to implement the proposed
spec based on this patch.
[1] virtio-rdma
- proposal:
https://lore.kernel.org/all/20220511095900.343-1-xieyongji@xxxxxxxxxxxxx/T/
- presentation on kvm forum:
https://youtu.be/Qrhv6hC_YK4

Please feel free to comment and suggest.
> Frank Li
>
> >
> > To realize the function, this patchset has few changes and introduces a
> > new APIs to PCI EP framework related to virtio. Furthermore, it device
> > depends on the some patchtes that is discussing. Those depended patchset
> > are following:
> > - [PATCH 1/2] dmaengine: dw-edma: Fix to change for continuous transfer
> > link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> > ernel.org%2Fdmaengine%2F20221223022608.550697-1-
> > mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
> > 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> > 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> > wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > C%7C%7C&sdata=tIn0MHzEvrdxaC4KKTvTRvYXBzQ6MyrFa2GXpa3ePv0%3D&
> > reserved=0
> > - [RFC PATCH 0/3] Deal with alignment restriction on EP side
> > link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> > ernel.org%2Flinux-pci%2F20230113090350.1103494-1-
> > mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
> > 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> > 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> > wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > C%7C%7C&sdata=RLpnDiLwfqQd5QMXdiQyPVCkfOj8q2AyVeZOwWHvlsM%3
> > D&reserved=0
> > - [RFC PATCH v2 0/7] Introduce a vringh accessor for IO memory
> > link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> > ernel.org%2Fvirtualization%2F20230202090934.549556-1-
> > mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
> > 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> > 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> > wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > C%7C%7C&sdata=6jgY76BMSbvamb%2Fl3Urjt4Gcizeqon%2BZE5nPssc2kDA%
> > 3D&reserved=0
> >
> > About this patchset has 4 patches. The first of two patch is little changes
> > to virtio. The third patch add APIs to easily access virtio data structure
> > on PCIe Host side memory. The last one introduce a virtio-net EP device
> > function. Details are in commit respectively.
> >
> > Currently those network devices are testd using ping only. I'll add a
> > result of performance evaluation using iperf and etc to the future version
> > of this patchset.
> >
> > Shunsuke Mie (4):
> >   virtio_pci: add a definition of queue flag in ISR
> >   virtio_ring: remove const from vring getter
> >   PCI: endpoint: Introduce virtio library for EP functions
> >   PCI: endpoint: function: Add EP function driver to provide virtio net
> >     device
> >
> >  drivers/pci/endpoint/Kconfig                  |   7 +
> >  drivers/pci/endpoint/Makefile                 |   1 +
> >  drivers/pci/endpoint/functions/Kconfig        |  12 +
> >  drivers/pci/endpoint/functions/Makefile       |   1 +
> >  .../pci/endpoint/functions/pci-epf-vnet-ep.c  | 343 ++++++++++
> >  .../pci/endpoint/functions/pci-epf-vnet-rc.c  | 635 ++++++++++++++++++
> >  drivers/pci/endpoint/functions/pci-epf-vnet.c | 387 +++++++++++
> >  drivers/pci/endpoint/functions/pci-epf-vnet.h |  62 ++
> >  drivers/pci/endpoint/pci-epf-virtio.c         | 113 ++++
> >  drivers/virtio/virtio_ring.c                  |   2 +-
> >  include/linux/pci-epf-virtio.h                |  25 +
> >  include/linux/virtio.h                        |   2 +-
> >  include/uapi/linux/virtio_pci.h               |   2 +
> >  13 files changed, 1590 insertions(+), 2 deletions(-)
> >  create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet-ep.c
> >  create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet-rc.c
> >  create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet.c
> >  create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet.h
> >  create mode 100644 drivers/pci/endpoint/pci-epf-virtio.c
> >  create mode 100644 include/linux/pci-epf-virtio.h
> >
> > --
> > 2.25.1
>
Best,
Shunsuke