Re: [BUG] kernel freezes with latest tree

[Ingo Molnar]

* Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:

> On Wed, 2012-01-11 at 16:56 +0100, Ingo Molnar wrote:
> 
> > Well, what happens if every CPU runs load_balance() and we keep 
> > triggering:
> > 
> >                 if (loops++ > sysctl_sched_nr_migrate) {
> >                         *lb_flags |= LBF_NEED_BREAK;
> >                         break;
> >                 }
> > 
> > in this case load_balance() will do the retry:
> > 
> >                 if (lb_flags & LBF_NEED_BREAK) {
> >                         lb_flags &= ~LBF_NEED_BREAK;
> >                         goto redo;
> >                 }
> > 
> > but the retry starts the loop again:
> > 
> >         list_for_each_entry_safe(p, n, &busiest_cfs_rq->tasks, se.group_node) {
> > 
> > so nobody is able to make progress: livelock/lockup.
> 
> Ah, right! Silly me. One possibility is to rotate that list, except that
> won't work for the cgroup case where we have another iteration.
> 
> OK, here's an updated patch.. 
> 
> ---
> Subject: sched: Limit load-balance retries on lock-break
> From: Peter Zijlstra <a.p.zijlstra@xxxxxxxxx>
> Date: Wed Jan 11 13:11:12 CET 2012
> 
> Eric and David reported dead machines and traced it to commit a195f004 ("sched:
> Fix load-balance lock-breaking"), it turns out there's still a
> scenario where we can end up re-trying forever.
> 
> Since there is no strict forward progress guarantee in the
> load-balance iteration we can get stuck re-retrying the same task-set
> over and over.
> 
> Creating a forward progress guarantee with the existing structure is
> somewhat non-trivial, for now simply terminate the retry loop after a
> few tries.
> 
> Reported-by: Eric Dumazet <eric.dumazet@xxxxxxxxx>
> Reported-by: David Ahern <dsahern@xxxxxxxxx>
> Signed-off-by: Peter Zijlstra <a.p.zijlstra@xxxxxxxxx>
> [eric: logic cleanup]
> Tested-by: Eric Dumazet <eric.dumazet@xxxxxxxxx>
> Link: http://lkml.kernel.org/n/tip-ya9m8grb9wfc26uqnviq2wjq@xxxxxxxxxxxxxx
> ---
>  kernel/sched/fair.c |   10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)

Thanks Peter, i'll get this fix to Linus ASAP.

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/