Re: [PATCH net-next v4 09/18] net/smc: introduce SMC-D loopback device

[Jan Karcher]

On 26/09/2023 09:24, Alexandra Winter wrote:
> On 25.09.23 17:18, Dust Li wrote:
> > > Hello Wen Gu,
> > >
> > > thank you for adding the Kconfig, so the distributions can decide when to offer this feature.
> > >
> > > I propose you add some kind of runtime switch as well. Not every user who loads the SMC module
> > > may want to exploit smcd-loopback. Especially in native environements without containers.
> > >
> > > If no RoCE interfaces or no ISM interfaces exist, the respective handling is skipped in SMC.
> > > If loopback is always created unconditionally, there is no way to opt-out.
> > Hi Sandy,
> >
> > After talking to Wen Gu offline, I think the real issue here might be
> > we don't have an abstract layer in SMC, something like net/core/dev.c
> >
> > Without this, we cannot do:
> >
> > 1. Enable/disable those devices dynamically
> >     Currently, If we want to disable a SMC-R device to communicate with
> >     others, we need to refer to 'ip link set dev xxx down' to disable the
> >     netdevice, then Infiniband subsystem will notify SMC that the state of
> >     the IB device has changed. We cannot explicitly choose not to use some
> >     specific IB/RoCE devices without disable totally.
> >     If the loopback device need to support enable/disable itself, I
> >     think it might be better to enable this feature for all SMC devices.
> >
> > 2. Do statistics per device
> >     Now, we have to relay on IB/RoCE devices' hardware statistics to see
> >     how many packets/bytes we have sent through this device.
> >
> > Both the above issues get worse when the IB/RoCE device is shared by SMC
> > and userspace RDMA applications. If SMC-R and userspace RDMA applications
> > run at the same time, we can't enable the device to run userspace RDMA
> > applications while block it from running SMC. For statistics, we cannot
> > tell how many packets/bytes were sent by SMC and how many were sent by
> > userspace RDMA applications.
> >
> > So I think those are better to support in the SMC layer.
> >
> > Best regards!
> > Dust
>
> Thank you very much for your considerations. I also think a generic handling
> of these requirements in the smc layer would be best. Especially, if we want
> to add virtio-ism support soon. There we will face the same issues again.
> Let's hear what others think about this.

Thanks you Sandy for bringing it up and Dust Li & Wen Gu for your thoughts.
I agree that such a runtime switch is needed and also that this generic 
handling would be good in the smc layer.