Re: [PATCH v3] iommu: Optimise PCI SAC address trick

From: Robin Murphy
Date: Thu Apr 13 2023 - 06:45:14 EST


On 2023-04-13 11:16, John Garry wrote:
On 20/01/2023 11:33, John Garry wrote:
On 18/01/2023 17:26, Robin Murphy wrote:
Per the reasoning in commit 4bf7fda4dce2 ("iommu/dma: Add config for
PCI SAC address trick") and its subsequent revert, this mechanism no
longer serves its original purpose, but now only works around broken
hardware/drivers in a way that is unfortunately too impactful to remove.

This does not, however prevent us from solving the performance impact
which the workaround imposes on large-scale systems that don't need it.
That is felt once the 32-bit IOVA space fills up and we keep
unsuccessfully trying to allocate from it. However, if we get to that
point then in fact it's already the endgame. The nature of the allocator
is such that the first IOVA we give to a device after the 32-bit space
runs out will be the highest possible address for that device, ever.
If that works, then great, we can be pretty sure it's safe to optimise
for speed by always allocating from the full range. And if it doesn't,
then the worst has already happened and any brokenness is now showing,
so there's no point continuing to try to hide it.

To that end, implement a flag to refine this into a per-device policy
that can automatically get itself out of the way if and when it stops
being useful.

CC: John Garry <john.garry@xxxxxxxxxx>
CC: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
Signed-off-by: Robin Murphy <robin.murphy@xxxxxxx>
---

v3: Expand the flag name, add a print with inline commentary for good
     measure, and refactor the code flow even more (too many ifs and
     indents...) such that I didn't presume to carry forward John's R-b.

I like the new changes, so feel free to add:

Reviewed-by: John Garry <john.g.garry@xxxxxxxxxx>

Is there any chance that this can be picked up?

I also saw that it fixed an issue for Jakub (cc'ed) recently.

Oh, thanks for the reminder - IIRC this wants a minor rebase now, let me double-check and send a v4 that applies cleanly...

Cheers,
Robin.