Re: [EXTERNAL] [tip: x86/apic] x86/io_apic: Cleanup trigger/polarity helpers

[Tom Lendacky]

On 11/10/20 10:33 AM, David Woodhouse wrote:
> On Tue, 2020-11-10 at 10:17 -0600, Tom Lendacky wrote:
>> Yep. The warning started triggering with:
>> 47bea873cf80 ("x86/msi: Only use high bits of MSI address for DMAR unit")
>>
>> Here's the backtrace:
>>
>> [   15.611109] ------------[ cut here ]------------
>> [   15.616274] WARNING: CPU: 184 PID: 1 at arch/x86/kernel/apic/apic.c:2505 __irq_msi_compose_msg+0x79/0x80
>> [   15.626855] Modules linked in:
>> [   15.630263] CPU: 184 PID: 1 Comm: swapper/0 Not tainted 5.10.0-rc1-sos-custom #1
>> [   15.638516] Hardware name: AMD Corporation ETHANOL_X/ETHANOL_X, BIOS REX1006G 01/25/2020
>> [   15.647549] RIP: 0010:__irq_msi_compose_msg+0x79/0x80
>> [   15.653188] Code: 0f f0 ff 09 d0 89 06 8b 47 04 c7 46 04 00 00 00 00 88 46 08 45 84 c0 74 08 8b 07 30 c0 89 46 04 c3 81 3f ff 00 00 00 77 01 c3 <0f> 0b c3 0f 1f 40 00 0f 1f 44 00 00 8b 05 81 f9 04 02 85 c0 75 16
>> [   15.674140] RSP: 0018:ffffc900000c7c30 EFLAGS: 00010212
>> [   15.679971] RAX: 0000000000000021 RBX: ffff888143789028 RCX: 0000000000000000
>> [   15.687936] RDX: 0000000000000000 RSI: ffffc900000c7c40 RDI: ffff8881447dd1c0
>> [   15.695899] RBP: ffff888143789028 R08: 0000000000000000 R09: 0000000000000005
>> [   15.703861] R10: 0000000000000002 R11: 0000000000000004 R12: ffff88900e7b80c0
>> [   15.711825] R13: 0000000000000000 R14: ffff88907646ba80 R15: 0000000000000000
>> [   15.719789] FS:  0000000000000000(0000) GS:ffff88900f000000(0000) knlGS:0000000000000000
>> [   15.728821] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [   15.735234] CR2: 0000000000000000 CR3: 000000114d40a000 CR4: 0000000000350ee0
>> [   15.743197] Call Trace:
>> [   15.745929]  irq_chip_compose_msi_msg+0x2e/0x40
>> [   15.750984]  msi_domain_activate+0x4b/0x90
>> [   15.755556]  __irq_domain_activate_irq+0x53/0x80
>> [   15.760707]  ? irq_set_msi_desc_off+0x5a/0x90
>> [   15.765568]  irq_domain_activate_irq+0x25/0x40
>> [   15.770525]  __msi_domain_alloc_irqs+0x16a/0x310
>> [   15.775680]  __pci_enable_msi_range+0x182/0x2b0
>> [   15.780738]  ? e820__memblock_setup+0x7d/0x7d
>> [   15.785597]  pci_enable_msi+0x16/0x30
>> [   15.789685]  iommu_init_msi+0x30/0x190
>> [   15.793860]  state_next+0x39d/0x665
>> [   15.797754]  ? e820__memblock_setup+0x7d/0x7d
>> [   15.802615]  iommu_go_to_state+0x24/0x28
>> [   15.806993]  amd_iommu_init+0x11/0x46
>> [   15.811076]  pci_iommu_init+0x16/0x3f
> 
> Oh joy.
> 
> It's asking the core code to generate a PCI MSI message for it and
> actually program that to the PCI device (since the IOMMU itself is a
> PCI device).
> 
> That isn't actually used for generating MSI, but is instead interpreted
> to write the intcapxt registers which *do* generate the interrupts.
> 
> That wants fixing, preferably not to go via MSI format at all, or maybe
> just to use the 'dmar' flag to __irq_msi_compose_msg(). Either way by
> having an irqdomain of its own like the Intel IOMMU does.
> 
> If I could get post-5.5 kernels to boot at all with the AMD IOMMU
> enabled, I'd have a go at throwing that together now...

That's odd. I haven't had an issue with or heard of any issues with
booting post-5.5 kernels with the AMD IOMMU enabled. What kind of problems
are you encountering?

Thanks,
Tom

>