Re: Re: [V2 PATCH 1/3] x86/panic: Fix re-entrance problem due to panic on NMI

[Michal Hocko]

On Wed 29-07-15 09:09:18, æåèå / KAWAIïHIDEHIRO wrote:
> > From: Michal Hocko [mailto:mhocko@xxxxxxxxxx]
> > On Wed 29-07-15 05:48:47, æåèå / KAWAIïHIDEHIRO wrote:
> > > Hi,
> > >
> > > > From: linux-kernel-owner@xxxxxxxxxxxxxxx [mailto:linux-kernel-owner@xxxxxxxxxxxxxxx] On Behalf Of Hidehiro Kawai
> > > > (2015/07/27 23:34), Michal Hocko wrote:
> > > > > On Mon 27-07-15 10:58:50, Hidehiro Kawai wrote:
> > > [...]
> > > > > The check could be also relaxed a bit and nmi_panic would
> > > > > return only if the ongoing panic is the current cpu when we really have
> > > > > to return and allow the preempted panic to finish.
> > > >
> > > > It's reasonable.  I'll do that in the next version.
> > >
> > > I noticed atomic_read() is insufficient.  Please consider the following
> > > scenario.
> > >
> > > CPU 1: call panic() in the normal context
> > > CPU 0: call nmi_panic(), check the value of panic_cpu, then call panic()
> > > CPU 1: set 1 to panic_cpu
> > > CPU 0: fail to set 0 to panic_cpu, then do an infinite loop
> > > CPU 1: call crash_kexec(), then call kdump_nmi_shootdown_cpus()
> > >
> > > At this point, since CPU 0 loops in NMI context, it never executes
> > > the NMI handler registered by kdump_nmi_shootdown_cpus().  This means
> > > that no register states are saved and no cleanups for VMX/SVM are
> > > performed.
> > 
> > Yes this is true but it is no different from the current state, isn't
> > it? So if you want to handle that then it deserves a separate patch.
> > It is certainly not harmful wrt. panic behavior.
> > 
> > > So, we should still use atomic_cmpxchg() in nmi_panic() to
> > > prevent other cpus from running panic routines.
> > 
> > Not sure what you mean by that.
> 
> I mean that we should use the same logic as my V2 patch like this:
> 
> #define nmi_panic(fmt, ...)                                            \
>        do {                                                            \
>                if (atomic_cmpxchg(&panic_cpu, -1, raw_smp_processor_id()) \
>                    == -1)                                              \
>                        panic(fmt, ##__VA_ARGS__);                      \
>        } while (0)

This would allow to return from NMI too eagerly. When I was testing my
previous approach (on 3.0 based kernel) I had basically the same thing
(one NMI to process panic) and others to return. This led to a strange
behavior when the NMI button triggered NMI on all (hundreds) CPUs. The
crash kernel booted eventually but the log contained lockups when a
CPU waited for an IPI to the CPU which was handling the NMI panic.

Anyway, I do not thing this is really necessary to solve the panic
reentrancy issue. If the missing saved state is a real problem then it
should be handled separately - maybe it can be achieved without an IPI
and directly from the panic context if we are in NMI.
-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/