Re: [PATCH] x86/tsc: RFC: re-synchronize TSCs to boot cpu TSC
From: Roland Scheidegger
Date: Fri Dec 09 2016 - 15:50:52 EST
Am 09.12.2016 um 18:33 schrieb Thomas Gleixner:
> On Fri, 9 Dec 2016, Roland Scheidegger wrote:
>> Am 09.12.2016 um 10:59 schrieb Thomas Gleixner:
>>> On Fri, 9 Dec 2016, Roland Scheidegger wrote:
>>>>
>>>> I saw some system lockups though:
>>>> When doing a cold boot, this kernel never managed to boot up. The last
>>>> message seen is:
>>>> x86: Booting SMP configuration:
>>>> .... node #0, CPUs: #1
>>>
>>> Weird. That really would be interesting to figure out what goes wrong
>>> there. What bothers me is that we don't see something like this:
>>>
>>>> [ 0.172334] TSC ADJUST differs: Reference CPU0: -577421768610 CPU1:
>>>> -577423766270
>>>
>>> Can you please apply the debug patch below and provide the output ?
>> Ok, this is the output (minus some typos maybe...):
>>
>> x86: Booting SMP configuration:
>> .... node #0, CPUs: #1
>> TSC ADJUST: CPU1: -2806491604
>> TSC source sync 0 -> 1 runs 3
>> TSC ADJUST differs: Reference CPU0: -2805503200 CPU1: -2806491604
>> TSC ADJUST synchronize: Reference CPU0: -2805503200 CPU1: -2806491604
>> TSC target sync skip
>> TSC source sync skipped
>>
>> And that's it.
>
> That's a cold boot (power on), right?
>
> That looks like I expected. As we know that the CPUs are in the same
> package simply resynchronizing the TSC ADJUST MSR is enough. And the sync
> test is skipped as we have resynced TSC ADJUST already to be the same
> value.
>
> What's unexpected is that the thing stops working :(
>
> I'm pretty sure the boot cpu hangs in that endless loop waiting for CPU1 to
> set the online bit, but for whatever reason this does not happen.
>
> Can you add the patch below to gather more information? There is a hunk in
> there with an '#if 0' which sets the TSC ADJUST to 0 on boot, which you can
> turn on as second step.
Ok, here's the results:
...
TSC ADJUST synchronize: Reference CPU0: -2820267100 CPU1: -2822498296
TSC target sync skipped
smpboot: Vector locked
smpboot: Vector setup done
smpboot: Clock setup
TSC source sync skipped
smpboot: Target CPU is online
With the #if 0 block activated, it boots up fine, the output was:
[ 1.038892] x86: Booting SMP configuration:
[ 1.038930] .... node #0, CPUs: #1
[ 0.171851] TSC ADJUST: CPU1: -2830353064 218577682002
[ 1.117495] TSC source sync 0 -> 1 runs 3
[ 0.171852] TSC ADJUST differs: Reference CPU0: -2828600940 CPU1:
-2830353064
[ 0.171853] TSC ADJUST synchronize: Reference CPU0: 0 CPU1: -2830353064
[ 1.117497] TSC target sync skip
[ 1.117497] smpboot: Vector locked
[ 1.117506] smpboot: Vector setup done
[ 1.117508] smpboot: Clock setup
[ 1.117510] smpboot: Enter idle startup
[ 1.117860] TSC source sync skipped
[ 1.117896] smpboot: Target CPU is online
[ 1.117990] #2
[ 0.252336] TSC ADJUST: CPU2: -2828948550 218818578394
[ 1.197511] TSC source sync 0 -> 2 runs 3
[ 0.252337] TSC ADJUST differs: Reference CPU0: -2828600940 CPU2:
-2828948550
[ 0.252338] TSC ADJUST synchronize: Reference CPU0: 0 CPU2: -2828948550
[ 1.197513] TSC target sync skip
[ 1.197513] smpboot: Vector locked
[ 1.197523] smpboot: Vector setup done
[ 1.197524] smpboot: Clock setup
[ 1.197527] smpboot: Enter idle startup
[ 1.197866] TSC source sync skipped
[ 1.197902] smpboot: Target CPU is online
[ 1.197997] #3
[ 0.332199] TSC ADJUST: CPU3: -2829409422 219057612986
[ 1.277528] TSC source sync 0 -> 3 runs 3
[ 0.332201] TSC ADJUST differs: Reference CPU0: -2828600940 CPU3:
-2829409422
[ 0.332202] TSC ADJUST synchronize: Reference CPU0: 0 CPU3: -2829409422
[ 1.277530] TSC target sync skip
[ 1.277530] smpboot: Vector locked
[ 1.277540] smpboot: Vector setup done
[ 1.277542] smpboot: Clock setup
[ 1.277544] smpboot: Enter idle startup
[ 1.277884] TSC source sync skipped
[ 1.277920] smpboot: Target CPU is online
[ 1.278016] #4
[ 0.411955] TSC ADJUST: CPU4: -2830194520 219296322250
[ 1.357546] TSC source sync 0 -> 4 runs 3
[ 0.411956] TSC ADJUST differs: Reference CPU0: -2828600940 CPU4:
-2830194520
[ 0.411957] TSC ADJUST synchronize: Reference CPU0: 0 CPU4: -2830194520
[ 1.357548] TSC target sync skip
[ 1.357548] smpboot: Vector locked
[ 1.357558] smpboot: Vector setup done
[ 1.357560] smpboot: Clock setup
[ 1.357562] smpboot: Enter idle startup
[ 1.357901] TSC source sync skipped
[ 1.357937] smpboot: Target CPU is online
[ 1.358030] #5
[ 0.491957] TSC ADJUST: CPU5: -2830235658 219535775620
[ 1.437562] TSC source sync 0 -> 5 runs 3
[ 0.491959] TSC ADJUST differs: Reference CPU0: -2828600940 CPU5:
-2830235658
[ 0.491960] TSC ADJUST synchronize: Reference CPU0: 0 CPU5: -2830235658
[ 1.437564] TSC target sync skip
[ 1.437565] smpboot: Vector locked
[ 1.437574] smpboot: Vector setup done
[ 1.437575] smpboot: Clock setup
[ 1.437578] smpboot: Enter idle startup
[ 1.437917] TSC source sync skipped
[ 1.437953] smpboot: Target CPU is online
[ 1.438047] #6
[ 0.571943] TSC ADJUST: CPU6: -2830332712 219775177676
[ 1.517581] TSC source sync 0 -> 6 runs 3
[ 0.571945] TSC ADJUST differs: Reference CPU0: -2828600940 CPU6:
-2830332712
[ 0.571946] TSC ADJUST synchronize: Reference CPU0: 0 CPU6: -2830332712
[ 1.517583] TSC target sync skip
[ 1.517583] smpboot: Vector locked
[ 1.517592] smpboot: Vector setup done
[ 1.517594] smpboot: Clock setup
[ 1.517597] smpboot: Enter idle startup
[ 1.517935] TSC source sync skipped
[ 1.517971] smpboot: Target CPU is online
[ 1.518066] #7
[ 0.652035] TSC ADJUST: CPU7: -2830107566 220014895498
[ 1.597597] TSC source sync 0 -> 7 runs 3
[ 0.652037] TSC ADJUST differs: Reference CPU0: -2828600940 CPU7:
-2830107566
[ 0.652037] TSC ADJUST synchronize: Reference CPU0: 0 CPU7: -2830107566
[ 1.597599] TSC target sync skip
[ 1.597600] smpboot: Vector locked
[ 1.597609] smpboot: Vector setup done
[ 1.597610] smpboot: Clock setup
[ 1.597613] smpboot: Enter idle startup
[ 1.597952] TSC source sync skipped
[ 1.597988] smpboot: Target CPU is online
[ 1.598083] #8
[ 0.732530] TSC ADJUST: CPU8: -2828717392 220255817924
[ 1.677628] TSC source sync 0 -> 8 runs 3
[ 0.732532] TSC ADJUST differs: Reference CPU0: -2828600940 CPU8:
-2828717392
[ 0.732532] TSC ADJUST synchronize: Reference CPU0: 0 CPU8: -2828717392
[ 1.677630] TSC target sync skip
[ 1.677630] smpboot: Vector locked
[ 1.677643] smpboot: Vector setup done
[ 1.677644] smpboot: Clock setup
[ 1.677647] smpboot: Enter idle startup
[ 1.677984] TSC source sync skipped
[ 1.678020] smpboot: Target CPU is online
[ 1.678123] #9
[ 0.812034] TSC ADJUST: CPU9: -2830240300 220493781182
[ 1.757641] TSC source sync 0 -> 9 runs 3
[ 0.812036] TSC ADJUST differs: Reference CPU0: -2828600940 CPU9:
-2830240300
[ 0.812036] TSC ADJUST synchronize: Reference CPU0: 0 CPU9: -2830240300
[ 1.757642] TSC target sync skip
[ 1.757643] smpboot: Vector locked
[ 1.757652] smpboot: Vector setup done
[ 1.757654] smpboot: Clock setup
[ 1.757656] smpboot: Enter idle startup
[ 1.757996] TSC source sync skipped
[ 1.758032] smpboot: Target CPU is online
[ 1.758130] #10
[ 0.892439] TSC ADJUST: CPU10: -2829079162 220734436186
[ 1.837658] TSC source sync 0 -> 10 runs 3
[ 0.892441] TSC ADJUST differs: Reference CPU0: -2828600940 CPU10:
-2829079162
[ 0.892441] TSC ADJUST synchronize: Reference CPU0: 0 CPU10: -2829079162
[ 1.837659] TSC target sync skip
[ 1.837660] smpboot: Vector locked
[ 1.837669] smpboot: Vector setup done
[ 1.837671] smpboot: Clock setup
[ 1.837673] smpboot: Enter idle startup
[ 1.838023] TSC source sync skipped
[ 1.838059] smpboot: Target CPU is online
[ 1.838155] #11
[ 0.972104] TSC ADJUST: CPU11: -2830133522 220972875474
[ 1.917674] TSC source sync 0 -> 11 runs 3
[ 0.972105] TSC ADJUST differs: Reference CPU0: -2828600940 CPU11:
-2830133522
[ 0.972106] TSC ADJUST synchronize: Reference CPU0: 0 CPU11: -2830133522
[ 1.917676] TSC target sync skip
[ 1.917677] smpboot: Vector locked
[ 1.917685] smpboot: Vector setup done
[ 1.917686] smpboot: Clock setup
[ 1.917689] smpboot: Enter idle startup
[ 1.918039] TSC source sync skipped
[ 1.918075] smpboot: Target CPU is online
[ 1.918171] #12
[ 1.052297] TSC ADJUST: CPU12: -2829606124 221212897186
[ 1.997691] TSC source sync 0 -> 12 runs 3
[ 1.052298] TSC ADJUST differs: Reference CPU0: -2828600940 CPU12:
-2829606124
[ 1.052299] TSC ADJUST synchronize: Reference CPU0: 0 CPU12: -2829606124
[ 1.997693] TSC target sync skip
[ 1.997693] smpboot: Vector locked
[ 1.997702] smpboot: Vector setup done
[ 1.997704] smpboot: Clock setup
[ 1.997706] smpboot: Enter idle startup
[ 1.998057] TSC source sync skipped
[ 1.998093] smpboot: Target CPU is online
[ 1.998189] #13
[ 1.132267] TSC ADJUST: CPU13: -2829742434 221452252884
[ 2.077708] TSC source sync 0 -> 13 runs 3
[ 1.132269] TSC ADJUST differs: Reference CPU0: -2828600940 CPU13:
-2829742434
[ 1.132270] TSC ADJUST synchronize: Reference CPU0: 0 CPU13: -2829742434
[ 2.077709] TSC target sync skip
[ 2.077710] smpboot: Vector locked
[ 2.077719] smpboot: Vector setup done
[ 2.077720] smpboot: Clock setup
[ 2.077722] smpboot: Enter idle startup
[ 2.078073] TSC source sync skipped
[ 2.078109] smpboot: Target CPU is online
[ 2.078203] #14
[ 1.212227] TSC ADJUST: CPU14: -2829914872 221691574212
[ 2.157724] TSC source sync 0 -> 14 runs 3
[ 1.212228] TSC ADJUST differs: Reference CPU0: -2828600940 CPU14:
-2829914872
[ 1.212229] TSC ADJUST synchronize: Reference CPU0: 0 CPU14: -2829914872
[ 2.157726] TSC target sync skip
[ 2.157726] smpboot: Vector locked
[ 2.157735] smpboot: Vector setup done
[ 2.157737] smpboot: Clock setup
[ 2.157740] smpboot: Enter idle startup
[ 2.159900] TSC source sync skipped
[ 2.159936] smpboot: Target CPU is online
[ 2.160032] #15
[ 1.292113] TSC ADJUST: CPU15: -2830310478 221930677086
[ 2.237743] TSC source sync 0 -> 15 runs 3
[ 1.292115] TSC ADJUST differs: Reference CPU0: -2828600940 CPU15:
-2830310478
[ 1.292115] TSC ADJUST synchronize: Reference CPU0: 0 CPU15: -2830310478
[ 2.237745] TSC target sync skip
[ 2.237745] smpboot: Vector locked
[ 2.237755] smpboot: Vector setup done
[ 2.237756] smpboot: Clock setup
[ 2.237759] smpboot: Enter idle startup
[ 2.238112] TSC source sync skipped
[ 2.238148] smpboot: Target CPU is online
[ 2.238186] x86: Booted up 1 node, 16 CPUs
[ 2.238224] smpboot: Total of 16 processors activated (95793.84 BogoMIPS)
(And fwiw with my quick hack the lockups disappear to when I change that
back to blast a zero into TSC_ADJ for all cpus.)
The system also came back up fine from suspend with this (well - still
minus graphics...), however disabled tsc clocksource:
[ 579.931739] Enabling non-boot CPUs ...
[ 579.943106] x86: Booting SMP configuration:
[ 579.943107] smpboot: Booting Node 0 Processor 1 APIC 0x2
[ 579.943189] TSC ADJUST: CPU1: -1504429974 21601834126
[ 579.943837] TSC source sync 0 -> 1 runs 3
[ 579.943191] TSC ADJUST differs: Reference CPU0: -2828600940 CPU1:
-1504429974
[ 579.943191] TSC ADJUST synchronize: Reference CPU0: 0 CPU1: -1504429974
[ 580.445833] TSC target sync skip
[ 580.445834] smpboot: Vector locked
[ 579.943839] TSC source sync skipped
[ 580.445845] smpboot: Vector setup done
[ 579.943850] smpboot: Target CPU is online
[ 580.445847] smpboot: Clock setup
[ 580.445852] smpboot: Enter idle startup
[ 580.445914] cache: parent cpu1 should not be sleeping
[ 579.944093] CPU1 is up
[ 580.458983] clocksource: timekeeping watchdog on CPU1: Marking
clocksource 'tsc' as unstable because the skew is too large:
[ 580.458985] clocksource: 'hpet' wd_now: 587c1
wd_last: 437c7 mask: ffffffff
[ 580.458986] clocksource: 'tsc' cs_now:
563963cd8 cs_last: 508f5a02a mask: ffffffffffffffff
[ 579.959249] smpboot: Booting Node 0 Processor 2 APIC 0x4
[ 579.959454] TSC ADJUST: CPU2: -1504299836 21650515856
[ 579.960058] TSC source sync 0 -> 2 runs 3
[ 579.959456] TSC ADJUST differs: Reference CPU0: -2828600940 CPU2:
-1504299836
[ 579.959456] TSC ADJUST synchronize: Reference CPU0: 0 CPU2: -1504299836
[ 580.462055] TSC target sync skip
[ 579.960060] TSC source sync skipped
[ 580.462056] smpboot: Vector locked
[ 580.462067] smpboot: Vector setup done
[ 579.960072] smpboot: Target CPU is online
[ 580.462068] smpboot: Clock setup
[ 580.462071] smpboot: Enter idle startup
[ 580.462136] cache: parent cpu2 should not be sleeping
[ 579.960327] CPU2 is up
[ 580.503230] smpboot: Booting Node 0 Processor 3 APIC 0x6
[ 580.503782] TSC ADJUST: CPU3: -1504357188 23279711288
[ 580.504405] TSC source sync 0 -> 3 runs 3
[ 580.503784] TSC ADJUST differs: Reference CPU0: -2828600940 CPU3:
-1504357188
[ 580.503784] TSC ADJUST synchronize: Reference CPU0: 0 CPU3: -1504357188
[ 581.006402] TSC target sync skip
[ 581.006403] smpboot: Vector locked
[ 580.504408] TSC source sync skipped
[ 581.006416] smpboot: Vector setup done
[ 580.504421] smpboot: Target CPU is online
[ 581.006418] smpboot: Clock setup
[ 581.006424] smpboot: Enter idle startup
[ 581.006506] cache: parent cpu3 should not be sleeping
[ 580.504741] CPU3 is up
[ 581.006760] [Firmware Bug]: TSC ADJUST differs: CPU0 0 -->
-1502494750. Restoring
[ 581.019271] smpboot: Booting Node 0 Processor 4 APIC 0x8
[ 580.517802] TSC ADJUST: CPU4: -1504419274 23321675136
[ 581.020442] TSC source sync 0 -> 4 runs 3
[ 580.517804] TSC ADJUST differs: Reference CPU0: -2828600940 CPU4:
-1504419274
[ 580.517805] TSC ADJUST synchronize: Reference CPU0: 0 CPU4: -1504419274
[ 581.020444] TSC target sync skip
[ 581.020444] TSC source sync skipped
[ 581.020445] smpboot: Vector locked
[ 581.020459] smpboot: Vector setup done
[ 581.020459] smpboot: Target CPU is online
[ 581.020461] smpboot: Clock setup
[ 581.020465] smpboot: Enter idle startup
[ 581.020548] cache: parent cpu4 should not be sleeping
[ 581.020785] CPU4 is up
[ 581.035383] smpboot: Booting Node 0 Processor 5 APIC 0xa
[ 580.534095] TSC ADJUST: CPU5: -1503892568 23370438110
[ 581.036558] TSC source sync 0 -> 5 runs 3
[ 580.534096] TSC ADJUST differs: Reference CPU0: -2828600940 CPU5:
-1503892568
[ 580.534097] TSC ADJUST synchronize: Reference CPU0: 0 CPU5: -1503892568
[ 581.036560] TSC target sync skip
[ 581.036560] TSC source sync skipped
[ 581.036561] smpboot: Vector locked
[ 581.036574] smpboot: Vector setup done
[ 581.036575] smpboot: Target CPU is online
[ 581.036576] smpboot: Clock setup
[ 581.036580] smpboot: Enter idle startup
[ 581.036664] cache: parent cpu5 should not be sleeping
[ 581.036913] CPU5 is up
[ 581.051460] smpboot: Booting Node 0 Processor 6 APIC 0xc
[ 580.550097] TSC ADJUST: CPU6: -1504107826 23418332356
[ 581.052632] TSC source sync 0 -> 6 runs 3
[ 580.550098] TSC ADJUST differs: Reference CPU0: -2828600940 CPU6:
-1504107826
[ 580.550099] TSC ADJUST synchronize: Reference CPU0: 0 CPU6: -1504107826
[ 581.052634] TSC target sync skip
[ 581.052634] smpboot: Vector locked
[ 581.052634] TSC source sync skipped
[ 581.052647] smpboot: Vector setup done
[ 581.052648] smpboot: Target CPU is online
[ 581.052650] smpboot: Clock setup
[ 581.052654] smpboot: Enter idle startup
[ 581.052738] cache: parent cpu6 should not be sleeping
[ 581.053002] CPU6 is up
[ 581.071544] smpboot: Booting Node 0 Processor 7 APIC 0xe
[ 580.570040] TSC ADJUST: CPU7: -1504535440 23478024002
[ 581.072718] TSC source sync 1 -> 7 runs 3
[ 580.570041] TSC ADJUST differs: Reference CPU0: -2828600940 CPU7:
-1504535440
[ 580.570042] TSC ADJUST synchronize: Reference CPU0: 0 CPU7: -1504535440
[ 581.072720] TSC target sync skip
[ 581.072720] smpboot: Vector locked
[ 581.072720] TSC source sync skipped
[ 581.072733] smpboot: Vector setup done
[ 581.072734] smpboot: Target CPU is online
[ 581.072736] smpboot: Clock setup
[ 581.072740] smpboot: Enter idle startup
[ 581.072822] cache: parent cpu7 should not be sleeping
[ 581.073092] CPU7 is up
[ 581.095281] smpboot: Booting Node 0 Processor 8 APIC 0x1
[ 580.594433] TSC ADJUST: CPU8: -1502571034 23551035204
[ 581.096455] TSC source sync 1 -> 8 runs 3
[ 580.594435] TSC ADJUST differs: Reference CPU0: -2828600940 CPU8:
-1502571034
[ 580.594436] TSC ADJUST synchronize: Reference CPU0: 0 CPU8: -1502571034
[ 581.096457] TSC target sync skip
[ 581.096457] smpboot: Vector locked
[ 581.096458] TSC source sync skipped
[ 581.096470] smpboot: Vector setup done
[ 581.096470] smpboot: Target CPU is online
[ 581.096472] smpboot: Clock setup
[ 581.096476] smpboot: Enter idle startup
[ 581.096593] cache: parent cpu8 should not be sleeping
[ 581.096881] CPU8 is up
[ 581.111327] smpboot: Booting Node 0 Processor 9 APIC 0x3
[ 580.609809] TSC ADJUST: CPU9: -1504556458 23597051168
[ 581.112494] TSC source sync 1 -> 9 runs 3
[ 580.609810] TSC ADJUST differs: Reference CPU0: -2828600940 CPU9:
-1504556458
[ 580.609811] TSC ADJUST synchronize: Reference CPU0: 0 CPU9: -1504556458
[ 581.112496] TSC target sync skip
[ 581.112496] TSC source sync skipped
[ 581.112496] smpboot: Vector locked
[ 581.112515] smpboot: Vector setup done
[ 581.112515] smpboot: Target CPU is online
[ 581.112518] smpboot: Clock setup
[ 581.112522] smpboot: Enter idle startup
[ 581.112635] cache: parent cpu9 should not be sleeping
[ 581.112919] CPU9 is up
[ 581.127367] smpboot: Booting Node 0 Processor 10 APIC 0x5
[ 580.626298] TSC ADJUST: CPU10: -1503281384 23646406682
[ 581.128558] TSC source sync 1 -> 10 runs 3
[ 580.626300] TSC ADJUST differs: Reference CPU0: -2828600940 CPU10:
-1503281384
[ 580.626301] TSC ADJUST synchronize: Reference CPU0: 0 CPU10: -1503281384
[ 581.128559] TSC target sync skip
[ 581.128560] TSC source sync skipped
[ 581.128560] smpboot: Vector locked
[ 581.128573] smpboot: Vector setup done
[ 581.128573] smpboot: Target CPU is online
[ 581.128575] smpboot: Clock setup
[ 581.128579] smpboot: Enter idle startup
[ 581.128689] cache: parent cpu10 should not be sleeping
[ 581.128997] CPU10 is up
[ 581.143754] smpboot: Booting Node 0 Processor 11 APIC 0x7
[ 580.642580] TSC ADJUST: CPU11: -1503566952 23695138352
[ 581.144935] TSC source sync 1 -> 11 runs 3
[ 580.642582] TSC ADJUST differs: Reference CPU0: -2828600940 CPU11:
-1503566952
[ 580.642583] TSC ADJUST synchronize: Reference CPU0: 0 CPU11: -1503566952
[ 581.144937] TSC target sync skip
[ 581.144937] smpboot: Vector locked
[ 581.144938] TSC source sync skipped
[ 581.144950] smpboot: Vector setup done
[ 581.144950] smpboot: Target CPU is online
[ 581.144952] smpboot: Clock setup
[ 581.144956] smpboot: Enter idle startup
[ 581.145069] cache: parent cpu11 should not be sleeping
[ 581.145493] CPU11 is up
[ 581.163769] smpboot: Booting Node 0 Processor 12 APIC 0x9
[ 580.662529] TSC ADJUST: CPU12: -1503752538 23754846788
[ 581.164945] TSC source sync 1 -> 12 runs 3
[ 580.662531] TSC ADJUST differs: Reference CPU0: -2828600940 CPU12:
-1503752538
[ 580.662531] TSC ADJUST synchronize: Reference CPU0: 0 CPU12: -1503752538
[ 581.164947] TSC target sync skip
[ 581.164948] TSC source sync skipped
[ 581.164949] smpboot: Vector locked
[ 581.164961] smpboot: Vector setup done
[ 581.164962] smpboot: Target CPU is online
[ 581.164963] smpboot: Clock setup
[ 581.164967] smpboot: Enter idle startup
[ 581.165084] cache: parent cpu12 should not be sleeping
[ 581.165564] CPU12 is up
[ 581.191866] smpboot: Booting Node 0 Processor 13 APIC 0xb
[ 580.690388] TSC ADJUST: CPU13: -1504470824 23838230778
[ 581.193045] TSC source sync 1 -> 13 runs 3
[ 580.690390] TSC ADJUST differs: Reference CPU0: -2828600940 CPU13:
-1504470824
[ 580.690391] TSC ADJUST synchronize: Reference CPU0: 0 CPU13: -1504470824
[ 581.193047] TSC target sync skip
[ 581.193047] smpboot: Vector locked
[ 581.193048] TSC source sync skipped
[ 581.193060] smpboot: Vector setup done
[ 581.193061] smpboot: Target CPU is online
[ 581.193063] smpboot: Clock setup
[ 581.193066] smpboot: Enter idle startup
[ 581.193184] cache: parent cpu13 should not be sleeping
[ 581.193637] CPU13 is up
[ 581.215919] smpboot: Booting Node 0 Processor 14 APIC 0xd
[ 580.714423] TSC ADJUST: CPU14: -1504515582 23910167978
[ 581.217095] TSC source sync 1 -> 14 runs 3
[ 580.714425] TSC ADJUST differs: Reference CPU0: -2828600940 CPU14:
-1504515582
[ 580.714426] TSC ADJUST synchronize: Reference CPU0: 0 CPU14: -1504515582
[ 581.217097] TSC target sync skip
[ 581.217097] smpboot: Vector locked
[ 581.217098] TSC source sync skipped
[ 581.217110] smpboot: Vector setup done
[ 581.217111] smpboot: Target CPU is online
[ 581.217113] smpboot: Clock setup
[ 581.217116] smpboot: Enter idle startup
[ 581.217234] cache: parent cpu14 should not be sleeping
[ 581.217706] CPU14 is up
[ 581.251975] smpboot: Booting Node 0 Processor 15 APIC 0xf
[ 580.750545] TSC ADJUST: CPU15: -1504326376 24018283718
[ 581.253154] TSC source sync 1 -> 15 runs 3
[ 580.750547] TSC ADJUST differs: Reference CPU0: -2828600940 CPU15:
-1504326376
[ 580.750548] TSC ADJUST synchronize: Reference CPU0: 0 CPU15: -1504326376
[ 581.253156] TSC target sync skip
[ 581.253156] TSC source sync skipped
[ 581.253157] smpboot: Vector locked
[ 581.253169] smpboot: Vector setup done
[ 581.253169] smpboot: Target CPU is online
[ 581.253171] smpboot: Clock setup
[ 581.253175] smpboot: Enter idle startup
[ 581.253294] cache: parent cpu15 should not be sleeping
[ 581.253774] CPU15 is up
[ 581.308954] ACPI: Waking up from system sleep state S3
[ 581.308984] clocksource: Switched to clocksource hpet
>
> This BIOS seems to be rather strange ...
Don't tell me...
No idea what it's doing, but I think it's safe to say whatever it's
trying to do, it's doing it wrong...
Roland