Re: [REGRESSION 5.19.x] AMD HD-audio devices missing on 5.19

From: Takashi Iwai
Date: Tue Aug 23 2022 - 11:47:09 EST


On Tue, 23 Aug 2022 08:06:05 +0200,
Takashi Iwai wrote:
>
> On Tue, 23 Aug 2022 03:00:21 +0200,
> Jason Gunthorpe wrote:
> >
> > On Mon, Aug 22, 2022 at 04:12:59PM +0200, Takashi Iwai wrote:
> > > Hi,
> > >
> > > we've received regression reports about the missing HD-audio devices
> > > on AMD platforms, and this turned out to be caused by the commit
> > > 512881eacfa72c2136b27b9934b7b27504a9efc2
> > > bus: platform,amba,fsl-mc,PCI: Add device DMA ownership management
> > >
> > > The details are found in openSUSE bugzilla:
> > > https://bugzilla.suse.com/show_bug.cgi?id=1202492
> > >
> > > The problem seems to be that HD-audio (both onboard analog and HDMI)
> > > PCI devices are assigned to the same IOMMU group as AMD graphics PCI
> > > device, and once after the AMDGPU is initialized beforehand, those
> > > audio devices can't be probed since iommu_device_use_default_domain()
> > > returns -EBUSY.
> >
> > Can you describe exactly what drivers are involved in this? If it is
> > the above commit then several devices are sharing an iommu group and
> > one of them (well, the only one already attached, I suppose) has made
> > the group unsharable.
> >
> > With grep I don't see an obvious place where the AMDGPU driver would
> > mess with the iommu configuration, so I have no guess.
>
> I have also no concrete clue, either :)
> At least, drivers/gpu/drm/amd/amdkfd/kfd_iommu.c calls
> amd_iommu_init_device(), and this invokes iommu_attach_group(), which
> may change group->domain. But it was just my wild guess, and it might
> be others, indeed.
>
> > It would be good to have some debugging to confirm if it is
> > group->owner (should be impossible, suggests memory corruption if it
> > is) or group->domain != group->default_domain.
> >
> > Most likely it is the later, but I can't see how that could happen on
> > a system like this.. There is no obvious manipulation in AMDGPU, for
> > instance.
> >
> > So debugging to find the backtrace for exactly when
> > group->domain != group->default_domain
> > Occurs for the troubled group would be necessary.
>
> OK, will try to build a test kernel with some debug prints and ask the
> reporters. It may take some time.

It was tested now and confirmed that the call path is via AMDGPU, as
expected:
amdgpu_pci_probe ->
amdgpu_driver_load_kms ->
amdgpu_device_init ->
amdgpu_amdkfd_device_init ->
kgd2kfd_device_init ->
kgd2kfd_resume_iommu ->
kfd_iommu_resume ->
amd_iommu_init_device ->
iommu_attach_group ->
__iommu_attach_group

At first AMDGPU driver is probed, and the iommu_attach_group() call
above changes the assigned group->domain. Afterwards, when HD-audio
devices are probed, it fails because:
- Both HD-audio PCI devices belong to the very same IOMMU group as the
AMD graphics PCI device
- PCI core calls iommu_device_use_default_domain() and the check
fails there because group->domain != group->default_domain

Takashi