Re: [GIT PULL] Networking for v6.9
From: Jakub Kicinski
Date: Tue Mar 12 2024 - 16:47:54 EST
On Tue, 12 Mar 2024 13:34:27 -0700 Jakub Kicinski wrote:
> I was testing with net-next on top of b0402403e54a ("Merge tag
> 'edac_updates_for_v6.9' of git://git.kernel.org/pub/scm/linux/kernel/git/ras/ras").
> IOW with the revert of 8e0ef4128694.
>
> I have various debug options enabled, and I only captured the stack
> trace from a kdump kernel, with hung_task_panic=1 which dies with
> the traces below. Let me retest now and try to capture the real
> crash..
With your tree as of 65d287c7eb1d it gets to prompt but dies soon after
when prod services kick in (dunno what rpm Kdump does but says iocost
so adding Tejun):
[ 115.832425] WARNING: CPU: 59 PID: 6473 at block/blk-iocost.c:2255 ioc_timer_fn+0x1dd4/0x2060
[ 115.851797] Modules linked in: tls act_gact cls_bpf kvm_amd kvm irqbypass acpi_cpufreq ipmi_si ipmi_devintf ipmi_msghandler button tpm_crb sch_fq_codel xhci_pci nvme xhci_hcd nvme_core fuse loop efivarfs autofs4 cbc cts ghash_generic gcm
[ 115.898730] CPU: 59 PID: 6473 Comm: rpm Kdump: loaded Not tainted 6.8.0-04854-gd99b04ab6218 #111
[ 115.918472] Hardware name: cheese!
[ 115.937746] RIP: 0010:ioc_timer_fn+0x1dd4/0x2060
[ 115.948369] Code: ff ff 0f 0b e9 04 f8 ff ff 48 8b 5c 24 10 48 8d bb f0 00 00 00 e8 1c 05 94 ff 48 c7 83 f0 00 00 00 00 00 00 00 e9 1b f7 ff ff <0f> 0b 48 8b bc 24 c0 00 00 00 e8 0d 35 a4 00 e9 97 f8 ff ff 48 8b
[ 115.990256] RSP: 0018:ffff888fced89ac8 EFLAGS: 00010046
[ 116.002156] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff81c354dc
[ 116.018379] RDX: dffffc0000000000 RSI: ffffffff82a86800 RDI: ffff8881612561e8
[ 116.034485] RBP: ffffe8ffffdc8000 R08: ffffffff81c31dbb R09: fffffbfff090ead6
[ 116.050519] R10: ffffffff848756b7 R11: 0000000000000000 R12: ffff888fced89cd8
[ 116.066741] R13: ffffe8ffffdc8010 R14: ffffe8ffffdc7fd0 R15: ffff888161256120
[ 116.083079] FS: 00007fae6679f780(0000) GS:ffff888fced80000(0000) knlGS:0000000000000000
[ 116.101394] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 116.114610] CR2: 00007f45dde301b0 CR3: 00000001a25cd002 CR4: 0000000000f70ef0
[ 116.131019] PKRU: 55555554
[ 116.137732] Call Trace:
[ 116.143788] <IRQ>
[ 116.148583] ? __warn+0xa0/0x230
[ 116.156470] ? ioc_timer_fn+0x1dd4/0x2060
[ 116.165729] ? report_bug+0x229/0x240
[ 116.174266] ? handle_bug+0x3c/0x70
[ 116.182311] ? exc_invalid_op+0x14/0x40
[ 116.191286] ? asm_exc_invalid_op+0x16/0x20
[ 116.200909] ? ioc_now+0x19b/0x240
[ 116.208973] ? ioc_timer_fn+0x3ac/0x2060
[ 116.218090] ? ioc_timer_fn+0x1dd4/0x2060
[ 116.227521] ? ioc_timer_fn+0x3ac/0x2060
[ 116.236616] ? transfer_surpluses+0xdc0/0xdc0
[ 116.246621] ? lock_acquire+0x184/0x440
[ 116.255451] ? lock_sync+0x110/0x110
[ 116.263832] ? __lock_acquire+0x8f2/0x3440
[ 116.273202] ? do_raw_spin_unlock+0x92/0xf0
[ 116.282808] ? transfer_surpluses+0xdc0/0xdc0
[ 116.292776] ? transfer_surpluses+0xdc0/0xdc0
[ 116.302815] call_timer_fn+0x13d/0x3a0
[ 116.311489] ? timer_shutdown_sync+0x10/0x10
[ 116.321782] ? lock_downgrade+0x3a0/0x3a0
[ 116.331131] ? transfer_surpluses+0xdc0/0xdc0
[ 116.341050] ? mark_held_locks+0x65/0x90
[ 116.350113] ? transfer_surpluses+0xdc0/0xdc0
[ 116.360151] __run_timers+0x445/0x570
[ 116.368667] ? call_timer_fn+0x3a0/0x3a0
[ 116.377777] ? lock_sync+0x110/0x110
[ 116.386245] ? do_raw_spin_lock+0x10f/0x1a0
[ 116.395904] ? spin_bug+0xe0/0xe0
[ 116.403775] run_timer_base+0x66/0x80
[ 116.412431] run_timer_softirq+0x16/0x30
[ 116.421505] __do_softirq+0x105/0x5de
[ 116.430031] irq_exit_rcu+0x94/0xf0
[ 116.438197] sysvec_apic_timer_interrupt+0x93/0xc0
[ 116.449178] </IRQ>
[ 116.454325] <TASK>
[ 116.459342] asm_sysvec_apic_timer_interrupt+0x16/0x20
[ 116.471096] RIP: 0010:_raw_spin_unlock_irqrestore+0x33/0x50
[ 116.483858] Code: 48 83 c7 18 53 48 89 f3 48 8b 74 24 10 e8 85 97 b6 fe 48 89 ef e8 ed e0 b6 fe 80 e7 02 74 06 e8 03 dc c9 fe fb bf 01 00 00 00 <e8> b8 b6 af fe 65 8b 05 59 1e 9c 7d 85 c0 74 03 5b 5d c3 e8 a5 96
[ 116.525813] RSP: 0018:ffff8881cf28f1d0 EFLAGS: 00000206
[ 116.537931] RAX: 0000000000034ad1 RBX: 0000000000000282 RCX: ffffffff811de87d
[ 116.554120] RDX: 0000000000000000 RSI: ffffffff82a86540 RDI: 0000000000000001
[ 116.570384] RBP: ffffffff8436c200 R08: 0000000000000001 R09: fffffbfff0c2dd7d
[ 116.586506] R10: 0000000000000001 R11: 0000000000000000 R12: ffff888241b3fc80
[ 116.602344] R13: 0000000000000080 R14: 0000000000000001 R15: 0000000000000282
[ 116.618234] ? mark_lock.part.0+0x10d/0x1790
[ 116.627775] __create_object+0x5e/0x80
[ 116.636199] kmem_cache_alloc+0x2f3/0x3b0
[ 116.645206] alloc_extent_state+0x1e/0x1a0
[ 116.654355] __set_extent_bit+0x32b/0xb30
[ 116.663374] lock_extent+0xc1/0x430
[ 116.671221] ? memcg_list_lru_alloc+0x580/0x580
[ 116.681348] ? try_lock_extent+0xf0/0xf0
[ 116.690125] ? preempt_count_sub+0x14/0xc0
[ 116.699283] ? folio_add_lru+0x1b6/0x2c0
[ 116.708079] ? filemap_add_folio+0xd0/0x130
[ 116.717417] ? __filemap_add_folio+0x7a0/0x7a0
[ 116.727382] add_ra_bio_pages.constprop.0.isra.0+0x294/0x650
[ 116.740055] btrfs_submit_compressed_read+0x339/0x490
[ 116.751363] ? btrfs_submit_compressed_write+0x2d0/0x2d0
[ 116.763251] ? lock_downgrade+0x3a0/0x3a0
[ 116.772238] submit_one_bio+0x6e/0x70
[ 116.780431] extent_readahead+0x9b4/0xa60
[ 116.789403] ? extent_readahead+0x19d/0xa60
[ 116.798755] ? extent_writepages+0x150/0x150
[ 116.808347] ? end_page_read+0x2a0/0x2a0
[ 116.817152] ? lockdep_hardirqs_on_prepare+0x220/0x220
[ 116.828633] ? __filemap_add_folio+0x439/0x7a0
[ 116.838619] ? folio_add_lru+0x1a7/0x2c0
[ 116.847426] ? lock_downgrade+0x3a0/0x3a0