Re: [PATCH RFC v2 rcu 2/8] srcu: Create an srcu_read_lock_nmisafe() and srcu_read_unlock_nmisafe()

From: Paul E. McKenney
Date: Mon Oct 03 2022 - 07:52:44 EST


On Mon, Oct 03, 2022 at 11:55:35AM +0200, Frederic Weisbecker wrote:
> On Sun, Oct 02, 2022 at 04:46:55PM -0700, Paul E. McKenney wrote:
> > On Sun, Oct 02, 2022 at 11:47:10PM +0200, Frederic Weisbecker wrote:
> > > On Sun, Oct 02, 2022 at 09:09:57AM -0700, Paul E. McKenney wrote:
> > > > On Sun, Oct 02, 2022 at 05:55:16PM +0200, Frederic Weisbecker wrote:
> > > > > On Thu, Sep 29, 2022 at 11:07:25AM -0700, Paul E. McKenney wrote:
> > > > > > @@ -1090,7 +1121,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
> > > > > > int ss_state;
> > > > > >
> > > > > > check_init_srcu_struct(ssp);
> > > > > > - idx = srcu_read_lock(ssp);
> > > > > > + idx = __srcu_read_lock_nmisafe(ssp);
> > > > >
> > > > > Why do we need to force the atomic based version here (even if CONFIG_NEED_SRCU_NMI_SAFE=y)?
> > > >
> > > > In kernels built with CONFIG_NEED_SRCU_NMI_SAFE=n, we of course need it.
> > > > As you say, in kernels built with CONFIG_NEED_SRCU_NMI_SAFE=y, we don't.
> > > > But it doesn't hurt to always use __srcu_read_lock_nmisafe() here, and
> > > > this is nowhere near a fastpath, so there is little benefit to using
> > > > __srcu_read_lock() when it is safe to do so.
> > > >
> > > > In addition, note that it is possible that a given srcu_struct structure's
> > > > first grace period is executed before its first reader. In that
> > > > case, we have no way of knowing which of __srcu_read_lock_nmisafe()
> > > > or __srcu_read_lock() to choose.
> > > >
> > > > So this code always does it the slow(ish) safe way.
> > >
> > > But then srcu_read_lock_nmisafe() would work as well, right?
> >
> > Almost.
> >
> > The problem is that without the leading "__", this would convince SRCU
> > that this is an NMI-safe srcu_struct. Which it might not be. Worse yet,
> > if this srcu_struct had already done an srcu_read_lock(), it would splat.
>
> Ah ok, now that makes sense.
>
> >
> > > > > > ss_state = smp_load_acquire(&ssp->srcu_size_state);
> > > > > > if (ss_state < SRCU_SIZE_WAIT_CALL)
> > > > > > sdp = per_cpu_ptr(ssp->sda, 0);
> > > > > > @@ -1123,7 +1154,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
> > > > > > srcu_funnel_gp_start(ssp, sdp, s, do_norm);
> > > > > > else if (needexp)
> > > > > > srcu_funnel_exp_start(ssp, sdp_mynode, s);
> > > > > > - srcu_read_unlock(ssp, idx);
> > > > > > + __srcu_read_unlock_nmisafe(ssp, idx);
> > > > > > return s;
> > > > > > }
> > > > > >
> > > > > > @@ -1427,13 +1458,13 @@ void srcu_barrier(struct srcu_struct *ssp)
> > > > > > /* Initial count prevents reaching zero until all CBs are posted. */
> > > > > > atomic_set(&ssp->srcu_barrier_cpu_cnt, 1);
> > > > > >
> > > > > > - idx = srcu_read_lock(ssp);
> > > > > > + idx = __srcu_read_lock_nmisafe(ssp);
> > > > >
> > > > > And same here?
> > > >
> > > > Yes, same here. ;-)
> > >
> > > Now bonus question: why do SRCU grace period starting/tracking
> > > need to be in an SRCU read side critical section? :o)
> >
> > Because I am lazy and like to keep things simple? ;-)
> >
> > More seriously, take a look at srcu_gp_start_if_needed() and the functions
> > it calls and ask yourself what bad things could happen if they were
> > preempted for an arbitrarily long period of time.
>
> I can see a risk for ssp->srcu_gp_seq to overflow. Can't say that was obvious
> though, at least for me. Am I missing something else?

That is what I recall. There might also be something else, of course. ;-)

Thanx, Paul