RE: [EXT] [RFC PATCH 0/3] Introduce a PCIe endpoint virtio console

[Frank Li]

> -----Original Message-----
> From: Shunsuke Mie <mie@xxxxxxxxxx>
> Sent: Sunday, April 16, 2023 9:12 PM
> To: Frank Li <frank.li@xxxxxxx>; Lorenzo Pieralisi <lpieralisi@xxxxxxxxxx>
> Cc: Krzysztof Wilczyński <kw@xxxxxxxxx>; Manivannan Sadhasivam
> <mani@xxxxxxxxxx>; Kishon Vijay Abraham I <kishon@xxxxxxxxxx>; Bjorn
> Helgaas <bhelgaas@xxxxxxxxxx>; Michael S. Tsirkin <mst@xxxxxxxxxx>;
> Jason Wang <jasowang@xxxxxxxxxx>; Jon Mason <jdmason@xxxxxxxx>;
> Randy Dunlap <rdunlap@xxxxxxxxxxxxx>; Ren Zhijie
> <renzhijie2@xxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx; linux-
> pci@xxxxxxxxxxxxxxx; virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
> Subject: Re: [EXT] [RFC PATCH 0/3] Introduce a PCIe endpoint virtio console
>
> Caution: EXT Email
>
> On 2023/04/14 23:39, Frank Li wrote:
> >
> >> -----Original Message-----
> >> From: Shunsuke Mie <mie@xxxxxxxxxx>
> >> Sent: Friday, April 14, 2023 7:39 AM
> >> To: Lorenzo Pieralisi <lpieralisi@xxxxxxxxxx>
> >> Cc: Krzysztof Wilczyński <kw@xxxxxxxxx>; Manivannan Sadhasivam
> >> <mani@xxxxxxxxxx>; Kishon Vijay Abraham I <kishon@xxxxxxxxxx>; Bjorn
> >> Helgaas <bhelgaas@xxxxxxxxxx>; Michael S. Tsirkin <mst@xxxxxxxxxx>;
> >> Jason Wang <jasowang@xxxxxxxxxx>; Shunsuke Mie <mie@xxxxxxxxxx>;
> >> Frank Li <frank.li@xxxxxxx>; Jon Mason <jdmason@xxxxxxxx>; Randy
> >> Dunlap <rdunlap@xxxxxxxxxxxxx>; Ren Zhijie <renzhijie2@xxxxxxxxxx>;
> >> linux-kernel@xxxxxxxxxxxxxxx; linux-pci@xxxxxxxxxxxxxxx;
> >> virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
> >> Subject: [EXT] [RFC PATCH 0/3] Introduce a PCIe endpoint virtio console
> >>
> >> Caution: EXT Email
> >>
> >> PCIe endpoint framework provides APIs to implement PCIe endpoint
> >> function.
> >> This framework allows defining various PCIe endpoint function behaviors
> in
> >> software. This patch extend the framework for virtio pci device. The
> >> virtio is defined to communicate guest on virtual machine and host side.
> >> Advantage of the virtio is the efficiency of data transfer and the
> conciseness
> >> of implementation device using software. It also be applied to PCIe
> >> endpoint function.
> >>
> >> We designed and implemented a PCIe EP virtio console function driver
> using
> >> the extended PCIe endpoint framework for virtio. It can be communicate
> >> host and endpoint over virtio as console.
> >>
> >> An architecture of the function driver is following:
> >>
> >>   ┌────────────┐         ┌────────────
> ──
> >> ────────┬────────────┐
> >>   │virtio      │         │                      │virtio      │
> >>   │console drv │         ├───────────────┐      │
> console
> >> drv │
> >>   ├────────────┤         │(virtio console│      ├───
> ──
> >> ───────┤
> >>   │ virtio bus │         │ device)       │◄────►│ virtio bus │
> >>   ├────────────┤         ├---------------┤      └────
> ──
> >> ──────┤
> >>   │            │         │ pci ep virtio │                   │
> >>   │  pci bus   │         │  console drv  │                   │
> >>   │            │  pcie   ├───────────────┤                   │
> >>   │            │ ◄─────► │  pci ep Bus   │                   │
> >>   └────────────┘         └────────────
> ──
> >> ─┴───────────────────┘
> >>     PCIe Root              PCIe Endpoint
> >>
> > [Frank Li] Some basic question,
> > I see you call register_virtio_device at epf_vcon_setup_vdev,
> > Why call it as virtio console?  I suppose it should be virtiobus directly?
>
> I'm sorry I didn't understand your question. What do you mean the
> virtiobus directly?

I go through your code again.  I think I understand why you need pci-epf-vcon.c.
Actually,  my means is like virtio_mmio_probe.

vm_dev->vdev.id.device = readl(vm_dev->base + VIRTIO_MMIO_DEVICE_ID);
vm_dev->vdev.id.vendor = readl(vm_dev->base + VIRTIO_MMIO_VENDOR_ID);

I am not sure that if VIRTIO_MMIO_VENDOR_ID and VIRTIO_MMIO_DEVICE_ID
reuse PCI's vendor ID and Device ID.  If yes, you can directly get such information
from epf.  If no,  a customer field can been added at epf driver.

So you needn't write pci-epf-vcon  and pci-epf-vnet .....

Of cause it will be wonderful if directly use virtio_mmio_probe by dynmatic create platform
Devices.  It may have some difficult because pci memory map requirement.

>
> >
> > Previous you use virtio-net, why change to virtio-console here?  Does it
> matter?
>
> No, it doesn't. Just I'd like to break down the changes into smaller
> steps to make it easier to review and merge the changes.
>
> As a first step, I propose adding a simplest virtio function driver with
> the extension defined in pci-epf-virtio.{h,c}.
>
> > All virtio-XXX should work?
> Yes, the extension is designed to use any type of virtio device.
> >
> > You removed EDMA support this version?
> I planed the support will be added with epf virtio-net patches.
> >
> >> Introduced driver is `pci ep virtio console drv` in the figure. It works
> >> as ep function for PCIe root and virtual virtio console device for PCIe
> >> endpoint. Each side of virtio console driver has virtqueue, and
> >> introduced driver transfers data on the virtqueue to each other. A data
> >> on root tx queue is transfered to endpoint rx queue and vice versa.
> >>
> >> This patchset is depend follwing patches which are under discussion.
> >>
> >> - [RFC PATCH 0/3] Deal with alignment restriction on EP side
> >> link:
> >>
> https://lore.k/
> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7Cff59a16f88c643913e3908db3
> ee91ca8%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C63817294315
> 3821831%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV
> 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hTB
> 5UDaJoJfta9ohMG%2BrxCVJY34ANn10iSLP9iCHX3M%3D&reserved=0
> >> ernel.org%2Flinux-pci%2F20230113090350.1103494-1-
> >>
> mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cea6513dbf
> >>
> 4084b80ced208db3ce54133%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7
> >>
> C0%7C638170727558800720%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
> >>
> LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C
> >> %7C%7C&sdata=jYgy%2Bxk84ZXZRVfqm0GCXoRnCTLMrX4zTfV%2Bs5Mm
> svo
> >> %3D&reserved=0
> >> - [RFC PATCH v2 0/7] Introduce a vringh accessor for IO memory
> >> link:
> >>
> https://lore.k/
> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7Cff59a16f88c643913e3908db3
> ee91ca8%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C63817294315
> 3821831%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV
> 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hTB
> 5UDaJoJfta9ohMG%2BrxCVJY34ANn10iSLP9iCHX3M%3D&reserved=0
> >> ernel.org%2Fvirtualization%2F20230202090934.549556-1-
> >>
> mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cea6513dbf
> >>
> 4084b80ced208db3ce54133%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7
> >>
> C0%7C638170727558800720%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
> >>
> LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C
> >> %7C%7C&sdata=K4El76GSAGtsWkNBXJK5%2Fn7flCN20eEMZpZYTX2WIZ0
> %3
> >> D&reserved=0
> >>
> >> First of this patchset is introduce a helper function to realize pci
> >> virtio function using PCIe endpoint framework. The second one is adding
> >> a missing definition for virtio pci header. The last one is for PCIe
> >> endpoint virtio console driver.
> >>
> >> This is tested on linux-20230406 and RCar S4 board as PCIe endpoint.
> >>
> >> Shunsuke Mie (3):
> >>    PCI: endpoint: introduce a helper to implement pci ep virtio function
> >>    virtio_pci: add a definition of queue flag in ISR
> >>    PCI: endpoint: Add EP function driver to provide virtio-console
> >>      functionality
> >>
> >>   drivers/pci/endpoint/functions/Kconfig        |  19 +
> >>   drivers/pci/endpoint/functions/Makefile       |   2 +
> >>   drivers/pci/endpoint/functions/pci-epf-vcon.c | 554
> ++++++++++++++++++
> >>   .../pci/endpoint/functions/pci-epf-virtio.c   | 469 +++++++++++++++
> >>   .../pci/endpoint/functions/pci-epf-virtio.h   | 123 ++++
> >>   include/uapi/linux/virtio_pci.h               |   3 +
> >>   6 files changed, 1170 insertions(+)
> >>   create mode 100644 drivers/pci/endpoint/functions/pci-epf-vcon.c
> >>   create mode 100644 drivers/pci/endpoint/functions/pci-epf-virtio.c
> >>   create mode 100644 drivers/pci/endpoint/functions/pci-epf-virtio.h
> >>
> >> --
> >> 2.25.1