RE: vPASID capability for VF

[Tian, Kevin]

> From: Alex Williamson <alex.williamson@xxxxxxxxxx>
> Sent: Thursday, May 11, 2023 1:25 AM
> 
> On Tue, 9 May 2023 08:34:53 +0000
> "Tian, Kevin" <kevin.tian@xxxxxxxxx> wrote:
> 
> > According to PCIe spec (7.8.9 PASID Extended Capability Structure):
> >
> >   The PASID configuration of the single non-VF Function representing
> >   the device is also used by all VFs in the device. A PF is permitted
> >   to implement the PASID capability, but VFs must not implement it.
> >
> > To enable PASID on VF then one open is where to locate the PASID
> > capability in VF's vconfig space. vfio-pci doesn't know which offset
> > may contain VF specific config registers. Finding such offset must
> > come from a device specific knowledge.
> 
> Backup for a moment, VFs are governed by the PASID capability on the
> PF.  The PASID capability exposes control registers that imply the
> ability to manage various feature enable bits.  The VF owner does not
> have privileges to manipulate those bits.  For example, the PASID Enable
> bit should restrict the endpoint from sending TLPs with a PASID prefix,
> but this can only be changed at the PF level for all associated VFs.
> 
> The protocol specified in 7.8.9.3 defines this enable bit as RW.  How do
> we virtualize that?  Either it's virtualized to be read-only and we
> violate the spec or we allow it to be read-write and it has no effect,
> which violates the spec.
> 

Currently the PASID cap is enabled by default when a device is probed
by iommu driver. Leaving it enabled in PF while guest wants it disabled
in VF is harmless. W/o proper setup in iommu side the VF cannot
do real work with PASID.

>From this angle fully virtualizing it in software looks good to me.

In another thread it's suggested that enabling the PASID cap should be
opted in by device driver instead of by iommu driver.

If that happens then vfio-pci may want to call into the PF driver
when the vPASID cap is enabled in VF. If the physical PASID cap in PF
hasn't been enabled then enable it. The PF driver will track which VF's
or its own clients require the PASID cap and keep it enabled until
no one wants it.