[Regression][BISECTED] kernel boot hang after 19898ce9cf8a ("wifi: iwlwifi: split 22000.c into multiple files")

From: Zhang, Rui
Date: Fri Jul 07 2023 - 04:25:28 EST


Hi, all,

I run into a NULL pointer dereference and kernel boot hang after
switching to latest upstream kernel, and git bisect shows that below
commit is the first offending commit, and I have confirmed that commit
19898ce9cf8a has the issue while 19898ce9cf8a~1 does not.

commit 19898ce9cf8a33e0ac35cb4c7f68de297cc93cb2 (refs/bisect/bad)
Author: Johannes Berg <johannes.berg@xxxxxxxxx>
AuthorDate: Wed Jun 21 13:12:07 2023 +0300
Commit: Johannes Berg <johannes.berg@xxxxxxxxx>
CommitDate: Wed Jun 21 14:07:00 2023 +0200

wifi: iwlwifi: split 22000.c into multiple files

Split the configuration list in 22000.c into four new files,
per new device family, so we don't have this huge unusable
file. Yes, this duplicates a few small things, but that's
still much better than what we have now.

Signed-off-by: Johannes Berg <johannes.berg@xxxxxxxxx>
Signed-off-by: Gregory Greenman <gregory.greenman@xxxxxxxxx>
Link:
https://lore.kernel.org/r/20230621130443.7543603b2ee7.Ia8dd54216d341ef1ddc0531f2c9aa30d30536a5d@changeid
Signed-off-by: Johannes Berg <johannes.berg@xxxxxxxxx>

I have some screenshots which show that RIP points to iwl_mem_free_skb,
I can create a kernel bugzilla and attach the screenshots there if
needed.

BTW, lspci output of the wifi device and git bisect log attached.

If any other information needed, please let me know.

thanks,
rui

00:14.3 Network controller: Intel Corporation Device 51f1 (rev 01)
Subsystem: Intel Corporation Device 0094
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 16
Region 0: Memory at 6001144000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [c8] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0
ExtTag- RBE- FLReset+
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr+ NoSnoop+ FLReset-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Via WAKE#, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis- LTR+ OBFF Disabled,
AtomicOpsCtl: ReqEn-
Capabilities: [80] MSI-X: Enable+ Count=16 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [100 v1] Latency Tolerance Reporting
Max snoop latency: 0ns
Max no snoop latency: 0ns
Capabilities: [164 v1] Vendor Specific Information: ID=0010 Rev=0 Len=014 <?>
Kernel driver in use: iwlwifi
Kernel modules: iwlwifi
00: 86 80 f1 51 06 04 10 00 01 00 80 02 00 00 80 00
10: 04 40 14 01 60 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 94 00
30: 00 00 00 00 c8 00 00 00 00 00 00 00 ff 01 00 00
40: 10 80 92 00 c0 0e 00 10 10 0c 10 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 12 08 08 00 05 04 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 11 00 0f 80 00 20 00 00 00 30 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 01 d0 23 c8 08 00 00 0d
d0: 05 40 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

git bisect start
# bad: [c17414a273b81fe4e34e11d69fc30cc8b1431614] Merge tag 'sh-for-v6.5-tag1' of git://git.kernel.org/pub/scm/linux/kernel/git/glaubitz/sh-linux
git bisect bad c17414a273b81fe4e34e11d69fc30cc8b1431614
# good: [6995e2de6891c724bfeb2db33d7b87775f913ad1] Linux 6.4
git bisect good 6995e2de6891c724bfeb2db33d7b87775f913ad1
# bad: [f8824e151fbfa0ac0a258015d606ea6f4a10251b] Merge tag 'sound-6.5-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound
git bisect bad f8824e151fbfa0ac0a258015d606ea6f4a10251b
# good: [6a8cbd9253abc1bd0df4d60c4c24fa555190376d] Merge tag 'v6.5-rc1-sysctl-next' of git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux
git bisect good 6a8cbd9253abc1bd0df4d60c4c24fa555190376d
# good: [08eeccb2491a3198f4adcba63adeace6e2499ea3] Merge tag 'linux-can-next-for-6.5-20230622' of git://git.kernel.org/pub/scm/linux/kernel/git/mkl/linux-can-next
git bisect good 08eeccb2491a3198f4adcba63adeace6e2499ea3
# bad: [e5476f57b32621eb8eab892a908df4d0b4808835] Merge tag 'gpio-updates-for-v6.5' of git://git.kernel.org/pub/scm/linux/kernel/git/brgl/linux
git bisect bad e5476f57b32621eb8eab892a908df4d0b4808835
# bad: [b545a13ca9b203eacce1e52ae2e32f99b860347c] Merge tag 'mlx5-updates-2023-06-21' of git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux
git bisect bad b545a13ca9b203eacce1e52ae2e32f99b860347c
# bad: [31aeae2446d50665b6ec51d564f5e7fe751d53d4] wifi: iwlwifi: cfg: remove trailing dash from FW_PRE constants
git bisect bad 31aeae2446d50665b6ec51d564f5e7fe751d53d4
# good: [108d202298bf03ce8d7e28bb94d23555c8385582] wifi: mac80211: use new inform_bss callback
git bisect good 108d202298bf03ce8d7e28bb94d23555c8385582
# good: [076031a09ae9e9394a56805aab92973612763d7e] wifi: rtw89: 8851b: configure to force 1 TX power value
git bisect good 076031a09ae9e9394a56805aab92973612763d7e
# good: [00e482010dfb5879fc9e8601d5819641fe4ae925] wifi: iwlwifi: mvm: add EHT A-MPDU size exponent support
git bisect good 00e482010dfb5879fc9e8601d5819641fe4ae925
# good: [e1374ed25324e87d675bda735841d8424d83c81d] wifi: iwlwifi: Add support for new CNVi (SC)
git bisect good e1374ed25324e87d675bda735841d8424d83c81d
# bad: [c648e926d021f9c6bdeef782df06f5111904fe7e] wifi: iwlwifi: don't load old firmware for 22000
git bisect bad c648e926d021f9c6bdeef782df06f5111904fe7e
# bad: [508b4a1baeb3da3f0bcf3ab4c94dd2c21cc5d395] wifi: iwlwifi: don't load old firmware for Sc
git bisect bad 508b4a1baeb3da3f0bcf3ab4c94dd2c21cc5d395
# bad: [5afe98b2e2995aa17ef77a29e57b1d98ccd6cd25] wifi: iwlwifi: give Sc devices their own family
git bisect bad 5afe98b2e2995aa17ef77a29e57b1d98ccd6cd25
# bad: [19898ce9cf8a33e0ac35cb4c7f68de297cc93cb2] wifi: iwlwifi: split 22000.c into multiple files
git bisect bad 19898ce9cf8a33e0ac35cb4c7f68de297cc93cb2
# first bad commit: [19898ce9cf8a33e0ac35cb4c7f68de297cc93cb2] wifi: iwlwifi: split 22000.c into multiple files