RE: [EXTERNAL] Re: [Patch v5 0/5] RDMA/mana_ib

[Long Li]

> > Do you have multiple "adapters" in one ib/eth device?
> > If yes, at least for IB, it will be very unexpected to see.
> >
> Adapter is IB specific and one per VF/pcie device. It's the handle that is passed
> between the management and VM for book keeping.

I think there are some confusions on the device.

There is one device, that supports both Ethernet and RDMA(through RoCE).

On the device, RoCE can't function without Ethernet. In the Linux, the RDMA device is modeled as auxiliary device to ethernet device.

The physical device exposes both Ethernet and RDMA management interfaces (as adapters) to the client (Linux kernel). When the MANA RDMA driver was first introduced, RAW QP was supported. There was no need to connect to the RDMA management interface. Many device hardware RDMA capabilities were hardcoded in the driver at the time. (There were some reviewers questioning this at the time).

With this patchset, we are connecting to the RDMA management interface on the device. This is for addressing the prior comments, and for supporting upcoming RC QP.

Thanks,

Long


> > Why do you have client and device when they are basically the same
> objects?
> >
> I am not sure which ones you are referring to specifically , can you please
> elaborate?
> 
> > > Please note that these are terms used for different purposes on the
> > management side.
> >
> > We are discussing RDMA side of this series.
> >
> > Thanks
> >
> > >
> > > > Thanks
> > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Leon Romanovsky <leon@xxxxxxxxxx>
> > > > > > Sent: Monday, September 11, 2023 7:33 AM
> > > > > > To: sharmaajay@xxxxxxxxxxxxxxxxx
> > > > > > Cc: Long Li <longli@xxxxxxxxxxxxx>; Jason Gunthorpe
> > > > > > <jgg@xxxxxxxx>; Dexuan Cui <decui@xxxxxxxxxxxxx>; Wei Liu
> > > > > > <wei.liu@xxxxxxxxxx>; David S. Miller <davem@xxxxxxxxxxxxx>;
> > > > > > Eric Dumazet <edumazet@xxxxxxxxxx>; Jakub Kicinski
> > > > > > <kuba@xxxxxxxxxx>; Paolo Abeni <pabeni@xxxxxxxxxx>; linux-
> > > > > > rdma@xxxxxxxxxxxxxxx; linux-hyperv@xxxxxxxxxxxxxxx;
> > > > > > netdev@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; Ajay
> > > > > > Sharma <sharmaajay@xxxxxxxxxxxxx>
> > > > > > Subject: [EXTERNAL] Re: [Patch v5 0/5] RDMA/mana_ib
> > > > > >
> > > > > > On Thu, Sep 07, 2023 at 09:52:34AM -0700,
> > > > > > sharmaajay@xxxxxxxxxxxxxxxxx
> > > > > > wrote:
> > > > > > > From: Ajay Sharma <sharmaajay@xxxxxxxxxxxxx>
> > > > > > >
> > > > > > > Change from v4:
> > > > > > > Send qp fatal error event to the context that created the qp.
> > > > > > > Add lookup table for qp.
> > > > > > >
> > > > > > > Ajay Sharma (5):
> > > > > > >   RDMA/mana_ib : Rename all mana_ib_dev type variables to
> > mib_dev
> > > > > > >   RDMA/mana_ib : Register Mana IB  device with Management SW
> > > > > > >   RDMA/mana_ib : Create adapter and Add error eq
> > > > > > >   RDMA/mana_ib : Query adapter capabilities
> > > > > > >   RDMA/mana_ib : Send event to qp
> > > > > >
> > > > > > I didn't look very deep into the series and has three very
> > > > > > initial
> > comments.
> > > > > > 1. Please do git log drivers/infiniband/hw/mana/ and use same
> > > > > > format for commit messages.
> > > > > > 2. Don't invent your own index-to-qp query mechanism in last
> > > > > > patch and use xarray.
> > > > > > 3. Once you decided to export mana_gd_register_device, it
> > > > > > hinted me that it is time to move to auxbus infrastructure.
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > >
> > > > > > >  drivers/infiniband/hw/mana/cq.c               |  12 +-
> > > > > > >  drivers/infiniband/hw/mana/device.c           |  81 +++--
> > > > > > >  drivers/infiniband/hw/mana/main.c             | 288 +++++++++++++-
> ----
> > > > > > >  drivers/infiniband/hw/mana/mana_ib.h          | 102 ++++++-
> > > > > > >  drivers/infiniband/hw/mana/mr.c               |  42 ++-
> > > > > > >  drivers/infiniband/hw/mana/qp.c               |  86 +++---
> > > > > > >  drivers/infiniband/hw/mana/wq.c               |  21 +-
> > > > > > >  .../net/ethernet/microsoft/mana/gdma_main.c   | 152 +++++----
> > > > > > >  drivers/net/ethernet/microsoft/mana/mana_en.c |   3 +
> > > > > > >  include/net/mana/gdma.h                       |  16 +-
> > > > > > >  10 files changed, 545 insertions(+), 258 deletions(-)
> > > > > > >
> > > > > > > --
> > > > > > > 2.25.1
> > > > > > >