Re: [PATCH v9 5/6] iommu/dma: Allow a single FQ in addition to per-CPU FQs

[Niklas Schnelle]

On Mon, 2023-05-15 at 10:07 -0300, Jason Gunthorpe wrote:
> On Mon, May 15, 2023 at 11:15:55AM +0200, Niklas Schnelle wrote:
> 
> > +/**
> > + * struct dma_iommu_options - Options for dma-iommu
> > + *
> > + * @flags: Flag bits for enabling/disabling dma-iommu settings
> > + *
> > + * This structure is intended to provide IOMMU drivers a way to influence the
> > + * behavior of the dma-iommu DMA API implementation. This allows optimizing for
> > + * example for a virtualized environment with slow IOTLB flushes.
> > + */
> > +struct dma_iommu_options {
> > +#define IOMMU_DMA_OPTS_PER_CPU_QUEUE	(0L << 0)
> > +#define IOMMU_DMA_OPTS_SINGLE_QUEUE	(1L << 0)
> > +	u64	flags;
> > +};
> 
> You need to hash it out with robin if we do something like this or use
> more untyped caps as he put in this series:
> 
> https://lore.kernel.org/linux-iommu/cover.1683233867.git.robin.murphy@xxxxxxx/
> 
> Jason

Ok. I do wonder how to best represent this as a capability.
Semantically I think a capability needs to be something positive i.e.
while IOMMU_CAP_EXPENSIVE_FLUSH would technically work having slow
IOTLB flushes really isn't a capability. So the best I can think of is
maybe IOMMU_CAP_SHADOW_ON_FLUSH. It's a bit specific but does convey
that the IOTLB flush does more than dropping hardware caches where the
main cost is the then empty TLB not the operation itself. Or maybe to
keep thing separate one would have to add capabilities for the existing
users IOMMU_CAP_HW_FLUSH and IOMMU_CAP_CONCURRENT_FLUSH.

Not sure though. It does feel more clunky than the tuning op I added
and maybe instead these mechanisms should co-exist. After all even
though the IOTLB flushes with shadowing are expensive they still
benefit from the flush queue just with more entries and less
parallelism.

Thanks,
Niklas