RE: [PATCH net-next v2 0/4] net/mlx5: Introduce devlink param to disable SF aux dev probe

[Parav Pandit]

> From: Jakub Kicinski <kuba@xxxxxxxxxx>
> Sent: Saturday, February 12, 2022 6:43 AM
> 
> On Fri, 11 Feb 2022 11:20:17 +0200 Moshe Shemesh wrote:
> > v1->v2:
> >  - updated example to make clear SF port and SF device creation PFs
> >  - added example when SF port and device creation PFs are on different
> > hosts
> 
> How does this address my comments?
>
Which one?
Your suggestion in [1] to specify vnet during port function spawning time?
Or 
Your suggestion in [2] to add "noprobe" option?

> We will not define Linux APIs based on what your firmware can or cannot do
> today. 
Sure. I just answered and clarified what it is the device capable to do.
We can possibly enhance the fw if it looks correct.

> Otherwise, why the global policy and all the hoops to jump thru?
> User wants a device with a vnet, give them a device with a vnet.
>
User wants a device with specific attributes.
Do you suggest to tunnel those params at port spawning time to share to different host via fw?
Some are HW/FW capabilities, and some are hints.
Some are hints because vnet also uses some eth resources by its very nature of being vnet.

Lets discuss two use cases.
Use case -1:
User wants params of [3] to below value.
eth=false, vnet=false, rdma=true, roce=false, io_eq_size=4, event_eq_size=256, max_macs=1.

Use case -2:
User wants params of [3] be below value.
eth=true, vnet=false, rdma=true, roce=true, rest=don't care.

Last year, when we added "roce" in [4] on the eswitch side, you commented in [4] to leave the decision on the SF side.
Based on this feedback, you can see growth of such params on the SF side in [5], [6] and reusing existing params in [7].
(instead of doing them on port spawning side)

Port spawning time attributes should cover minimum of below attributes of [3].
(a) enable_vnet,eth,roce,rdma,iwarp (b) io_eq_size, (c) event_eq_size (d) max_macs.

Do you agree if above list is worth addition as port function attributes?
If not, its not addressing the user needs.

Did you get a chance to read my reply in [8]?
In future when user wants to change the cpu affinity of a SF, user needs to perform devlink reload.
And params of [3] + any new devlink params also benefit from single devlink reload?
For example, which and how many cpus to use is something best decided by the depending on the workload and use case.

> You left out from your steps how ESW learns that the device has to be
> spawned. 
I read above note few times, but didn't understand. Can you please explain?

> Given there's some form of communication between user intent and
> ESW the location of the two is completely irrelevant.
I find it difficult to have all attributes on the port function, specially knobs which are very host specific.
Few valid knobs that I see on host side are 
(a) cpu affinity mask
(b) number of msix vectors to consume within driver internally vs map to user space

At present I see knobs on both sides.
Saeed is offline this week, and I want to gather his feedback as well on passing hints from port spawning side to host side.