Re: printk.time causes rare kernel boot hangs

From: Peter Zijlstra
Date: Wed Jun 14 2023 - 05:22:11 EST


On Tue, Jun 13, 2023 at 02:41:05PM +0100, Richard W.M. Jones wrote:
> [Being tracked in this bug which contains much more detail:
> https://gitlab.com/qemu-project/qemu/-/issues/1696 ]

Can I please just get the detail in mail instead of having to go look at
random websites?

> Recent kernels hang rarely when booted on qemu. Usually you need to
> boot 100s or 1,000s of times to see the hang, compared to 292,612 [sic]
> successful boots which I was able to do before the problematic commit.
>
> A reproducer (you'll probably need to use Fedora) is:

Debian only shop here... in fact, I still have machines without systemd.

> $ while guestfish -a /dev/null -v run >& /tmp/log; do echo -n . ; done
>
> You will need to leave it running for probably several hours, and
> examine the /tmp/log file at the end.
>
> I tracked this down to the following commit:
>
> commit f31dcb152a3d0816e2f1deab4e64572336da197d
> Author: Aaron Thompson <dev@xxxxxxxxxx>
> Date: Thu Apr 13 17:50:12 2023 +0000
>
> sched/clock: Fix local_clock() before sched_clock_init()
>
> Have local_clock() return sched_clock() if sched_clock_init() has not
> yet run. sched_clock_cpu() has this check but it was not included in the
> new noinstr implementation of local_clock().
>
> (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f31dcb152a3d0816e2f1deab4e64572336da197d)
>
> Reverting this commit fixes the problem.
>
> I don't know _why_ this commit is wrong, but can we revert it as it
> causes serious problems with libguestfs hanging randomly.
>
> Or if there's anything you want me to try out then let me know,
> because I can reproduce the problem locally quite easily.

Well, since it's virt and all, can you attach gdb to the gdb-stub and
see where it's at? Any clue is better than no clue.