Re: [PATCH v2 0/5] Optimize iommu_map_sg() performance
From: Robin Murphy
Date: Tue Jan 12 2021 - 21:55:21 EST
On 2021-01-12 16:33, Christoph Hellwig wrote:
On Tue, Jan 12, 2021 at 04:00:59PM +0000, Robin Murphy wrote:
Out of curiosity, how much of the difference is attributable to actual
indirect call overhead vs. the additional massive reduction in visits to
arm_smmu_rpm_{get,put} that you fail to mention? There are ways to optimise
indirect calling that would benefit *all* cases, rather than just one
operation for one particular driver.
Do we have systems that use different iommu_ops at the same time?
If not this would be a prime candidate for static call optimizations.
They're not at all common, but such systems do technically exist. It's
hard to make them work in the current "one set of ops per bus" model,
but I still have a long-term dream of sorting that out so such setups
*can* be supported properly. I certainly wouldn't want to make any
changes that completely close the door on that idea, but any static call
optimisation that can be done in a config-gated manner should be viable
for x86 at least. Even better if we could do it with a dynamic
branch-patching solution that keeps the indirect case as a fallback;
AFAICS that should be feasible to eagerly apply somewhere around
iommu_device_register(), then just undo again if another driver ever
does show up registering a new set of ops that don't match. I'm pretty
confident that the systems where performance matters most are going to
be sensible homogeneous ones - on the Arm side the SBSA should see to
that. The weird mix-and-match cases are typically going to be FPGA
prototyping systems and esoteric embedded stuff that are worlds away
from worrying about keeping up with line rate on a 40GbE NIC...
Also I've been pondering adding direct calls to the iommu dma ops like
we do for DMA direct. This would allow to stop using dma_ops entirely
for arm64.
Yes, now that we're starting to get things sufficiently consolidated
behind iommu-dma that might be a reasonable thing to try, although given
the amount of inherent work further down in the IOVA and IOMMU layers
that dwarfs that of the direct case, I doubt that reducing the initial
dispatch overhead would make any noticeable difference in practice.
Robin.