Re: 2.6.33-rc1 unusable due to scheduler issues, circular locking, WARNs and BUGs

From: AmÃrico Wang
Date: Tue Dec 22 2009 - 02:50:37 EST


On Tue, Dec 22, 2009 at 3:41 PM, Xiaotian Feng <xtfeng@xxxxxxxxx> wrote:
> On Tue, Dec 22, 2009 at 3:19 PM, AmÃrico Wang <xiyou.wangcong@xxxxxxxxx> wrote:
>> [Fix top-posting]
>>
>> On Tue, Dec 22, 2009 at 1:42 PM, Xiaotian Feng <xtfeng@xxxxxxxxx> wrote:
>>>
>>> On Tue, Dec 22, 2009 at 8:17 AM, Eric Paris <eparis@xxxxxxxxxx> wrote:
>>>> Trying to build a kernel on a 48 core x86_64 box using make -j 64 and
>>>> I'm exploding in the scheduler. ÂI'm running (and building) kernel
>>>> f7b84a6ba7eaeba4e1df8feddca1473a7db369a5 ÂThere are three distinct
>>>> signatures of problems. ÂSome boots I'll see all 3 of these failures
>>>> sometimes only 1 or 2 of them. ÂThat's the reason they are kinda split
>>>> up in dmesg.
>>>>
>>>> 1) gcc/3141 is trying to acquire lock:
>>>> Â(&(&sem->wait_lock)->rlock){......}, at: [<ffffffff81223234>] __down_read_trylock+0x13/0x46
>>>>
>>>> but task is already holding lock:
>>>> Â(&rq->lock){-.-.-.}, at: [<ffffffff8103dd2d>] task_rq_lock+0x51/0x83
>>>>
>>>> 2) WARN() in kernel/sched_fair.c:1001 hrtick_start_fair()
>>>>
>>>> 3) NULL pointer dereference at 0000000000000168 in check_preempt_wakeup
>>>> Â Â Âkernel/sched_fair.c
>>>>
>>>> Full backtraces are in the attached dmesg.
>>>>
>>> Does a revert of cd29fe6f2637cc2ccbda5ac65f5332d6bf5fa3c6 fix this problem?
>>
>>
>> I don't think so...
>>
>> I think the most suspicious commit here is ab19cb23. It kicked
>> "local_irq_save()"
>> out, which means if the task is selected to run on another cpu which doesn't
>> disable irq, we will have a page fault, thun we will try to hold mm->mmap_sem
>> while we are holding rq->lock already.
>
> The page fault is from kernel ÂNULL pointer deref. ÂYou should connect
> the lockdep warning and kernel BUG together.
>

Interesting.

1) Doesn't this NULL ptr def expose that we have a potential problem?

2) For NULL ptr def problem, commit 3a7e73a2e2 seems more suspicious..
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/