Re: [ANNOUNCE] 3.7-nohz1

[Paul E. McKenney]

On Sat, Jan 05, 2013 at 12:42:53AM +0100, Frederic Weisbecker wrote:
> 2012/12/30 Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>:
> > On Mon, Dec 24, 2012 at 12:43:25AM +0100, Frederic Weisbecker wrote:
> >> 2012/12/21 Steven Rostedt <rostedt@xxxxxxxxxxx>:
> >> > On Thu, 2012-12-20 at 19:32 +0100, Frederic Weisbecker wrote:
> >> >> Let's imagine you have 4 CPUs. We keep the CPU 0 to offline RCU callbacks there and to
> >> >> handle the timekeeping. We set the rest as full dynticks. So you need the following kernel
> >> >> parameters:
> >> >>
> >> >>       rcu_nocbs=1-3 full_nohz=1-3
> >> >>
> >> >> (Note rcu_nocbs value must always be the same as full_nohz).
> >> >
> >> > Why? You can't have: rcu_nocbs=1-4 full_nohz=1-3
> >>
> >> That should be allowed.
> >>
> >> >   or: rcu_nocbs=1-3 full_nohz=1-4 ?
> >>
> >> But that not.
> >>
> >> You need to have: rcu_nocbs & full_nohz == full_nohz. This is because
> >> the tick is not there to maintain the local RCU callbacks anymore. So
> >> this must be offloaded to the rcu_nocb threads.
> >>
> >> I just have a doubt with rcu_nocb. Do we still need the tick to
> >> complete the grace period for local rcu callbacks? I need to discuss
> >> that with Paul.
> >
> > The tick is only needed if rcu_needs_cpu() returns false.  Of course,
> > this means that if you don't invoke rcu_needs_cpu() before returning to
> > adaptive-idle usermode execution, you are correct that a full_nohz CPU
> > would also have to be a rcu_nocbs CPU.
> >
> > That said, I am getting close to having an rcu_needs_cpu() that only
> > returns false if there are callbacks immediately ready to invoke, at
> > least if RCU_FAST_NO_HZ=y.
> 
> Ok. Also when a CPU enqueues a callback and starts a grace period, the
> tick polls on the grace period completion.

If RCU_FAST_NO_HZ=n, then yes, this is the case, but only for !rcu_nocbs
CPUs.

>                                            How is it handled with
> rcu_nocbs CPUs? Does rcu_needs_cpu() return false until the grace
> period is completed? If so I still need to restart the local tick
> whenever a new callback is enqueued.

Each rcu_nocbs CPU has a kthread, and that kthread is responsible for
making sure that any needed grace periods move forward.  In mainline, this
is done via CPU 0, which is required to be a !rcu_nocbs CPU.  In -rcu,
the no-CBs kthreads communicate with the grace-period kthread via the
rcu_node tree, so that if all CPUs are rcu_nocbs CPUs, rcu_needs_cpu()
will always return false, even if RCU_FAST_NO_HZ=n.

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/