Re: Should RCU_BOOST kernels use hrtimers in GP kthread?

[Paul E. McKenney]

On Wed, Feb 17, 2021 at 07:54:47AM -0800, Paul E. McKenney wrote:
> On Wed, Feb 17, 2021 at 04:32:53PM +0100, Sebastian Andrzej Siewior wrote:
> > On 2021-02-16 10:36:09 [-0800], Paul E. McKenney wrote:
> > > Hello, Sebastian,
> > 
> > Hi Paul,
> > 
> > > I punted on this for the moment by making RCU priority boosting testing
> > > depend on CONFIG_PREEMPT_RT, but longer term I am wondering if RCU's
> > > various timed delays and timeouts should use hrtimers rather than normal
> > > timers in kernels built with CONFIG_RCU_BOOST.  As it is, RCU priority
> > > boosting can be defeated if any of the RCU grace-period kthread's timeouts
> > > are serviced by the non-realtime ksoftirqd.
> > 
> > I though boosting is accomplished by acquiring a rt_mutex in a
> > rcu_read() section. Do you have some code to point me to, to see how a
> > timer is involved here? Or is it the timer saying that *now* boosting is
> > needed.
> 
> Yes, this last, which is in the grace-period kthread code, for example,
> in rcu_gp_fqs_loop().
> 
> > If your hrtimer is a "normal" hrtimer then it will be served by
> > ksoftirqd, too. You would additionally need one of the
> > HRTIMER_MODE_*_HARD to make it work.
> 
> Good to know.  Anything I should worry about for this mode?
> 
> Also, the current test expects callbacks to be invoked, which involves a
> number of additional kthreads and timers, for example, in nocb_gp_wait().
> I suppose I could instead look at grace-period sequence numbers, but I
> believe that real-life use cases needing RCU priority boosting also need
> the callbacks to be invoked reasonably quickly (as in within hundreds
> of milliseconds up through very small numbers of seconds).
> 
> Thoughts?

Hmmm...  Unless there are current use cases where callbacks are being
prevented from being invoked, I will modify rcutorture's testing of RCU
priority boosting to look only at grace-period progress on the theory
that most real-time uses offload callbacks, and in that case it is the
sysadm's job to make sure that they get the CPU time they needs.

							Thanx, Paul

> > > This might require things like swait_event_idle_hrtimeout_exclusive(),
> > > either as primitives or just open coded.
> > > 
> > > Thoughts?
> > > 
> > > 							Thanx, Paul
> > 
> > Sebastian