RE: vPASID capability for VF

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@xxxxxxxxxx>
> Sent: Wednesday, May 10, 2023 7:13 AM
> 
> On Tue, May 09, 2023 at 10:57:04PM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe <jgg@xxxxxxxxxx>
> > > Sent: Wednesday, May 10, 2023 6:44 AM
> > >
> > > On Tue, May 09, 2023 at 08:34:53AM +0000, Tian, Kevin wrote:
> > > > According to PCIe spec (7.8.9 PASID Extended Capability Structure):
> > > >
> > > >   The PASID configuration of the single non-VF Function representing
> > > >   the device is also used by all VFs in the device. A PF is permitted
> > > >   to implement the PASID capability, but VFs must not implement it.
> > > >
> > > > To enable PASID on VF then one open is where to locate the PASID
> > > > capability in VF's vconfig space. vfio-pci doesn't know which offset
> > > > may contain VF specific config registers. Finding such offset must
> > > > come from a device specific knowledge.
> > >
> > > Why? Can't vfio probe the cap tree and just find a gap to insert a new
> > > cap? We already mangle the cap list, I'm not sure I see what
> > > the problem is?
> > >
> >
> > PCI config space includes not only caps, but also device specific
> > defined fields. e.g. Intel IGD defines offset 0xfc as a pointer to
> > OpRegion. I'm sure Alex can give many other examples.
> 
> Do we even expose those over VIFO? I thought in general we blocked of

Yes. I did a quick check:

/*
 * Default unassigned regions to raw read-write access.  Some devices
 * require this to function as they hide registers between the gaps in
 * config space (be2net).  Like MMIO and I/O port registers, we have
 * to trust the hardware isolation.
 */
static struct perm_bits unassigned_perms = {
	.readfn = vfio_raw_config_read,
	.writefn = vfio_raw_config_write
};

vfio_config_do_rw()
{
	...
	if (cap_id == PCI_CAP_ID_INVALID) {
		perm = &unassigned_perms;
		cap_start = *ppos;
	} ...
}

vfio_config_init()
{
	...
	memset(map, PCI_CAP_ID_BASIC, PCI_STD_HEADER_SIZEOF);
	memset(map + PCI_STD_HEADER_SIZEOF, PCI_CAP_ID_INVALID,
              		 pdev->cfg_size - PCI_STD_HEADER_SIZEOF);
	...
}

> various parts of the config space. I keep seeing patches to unblock
> parts of config space?
> 
> I'd do the reverse and say devices that want to pass parts of their
> config space should have a special hook to do it and otherwise we
> should sanitize and block?

This then may break backward compatibility. We don't know how
many devices have such hidden registers so if anyone misses a hook
immediately it cannot be assigned after we start blocking as default.

> 
> eg we already have a hook to pass the opregion
> 
> > So it's easy to find the gap between caps, but not easy to know
> > whether that gap is actually free to use.
> 
> Because, let's face it, this is a horrible thing to do, and the
> opregion stuff is just ugly as s sin.
> 

It's ugly, but that is the reality. :/