softlockup during glibc tst-robust in 2.6.18-rc3

From: Olaf Hering
Date: Wed Aug 02 2006 - 01:36:11 EST


A 'make check' in glibc mainline triggers this lockup in current Linus tree on a
4cpu p630. All I can do at this point is a reboot via sysrq b.

According to the glibc build log, tst-robust1 and tst-robust2 timed out, tst-robust3
is still running.
I'm using pseries_defconfig with HZ=100.

vmstat 9:

procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
6 0 0 3594448 649356 3226392 0 0 0 9611 366 2636 41 21 32 6 0
2 0 0 3522548 650480 3290164 0 0 0 7151 285 3150 39 22 35 5 0
2 2 0 3483124 650904 3305464 0 0 60 4008 197 1883 43 12 38 7 0
5 1 0 3432532 651984 3347232 0 0 64 8288 372 1070 52 8 20 20 0
6 0 0 3351124 657576 3415872 0 0 0 7416 318 929 47 14 20 20 0
1 5 0 3311736 658716 3426652 0 0 0 3872 434 1541 57 18 16 8 0
2 2 0 3295604 661336 3446192 0 0 0 3957 313 749 22 37 22 18 0
5 0 0 3714908 661488 2996632 0 0 0 4282 428 738 53 16 25 6 0
6 0 0 3713136 661544 3017640 0 0 0 3763 622 1422 72 9 3 15 0
1 5 0 3621032 663432 3069864 0 0 0 8847 3173 6399 47 13 26 14 0
7 0 0 3591824 666688 3109752 0 0 0 5911 463 789 52 9 17 22 0
5 0 0 3467652 668708 3186068 0 0 0 4560 212 747 74 9 11 7 0
7 0 0 3434724 669512 3205616 0 0 0 7974 368 1022 51 16 16 17 0
0 7 0 3514044 670604 3221456 0 0 0 5403 273 1166 66 12 7 15 0
3 1 0 3926980 671000 2752576 0 0 0 4643 306 1226 45 28 16 11 0
5 1 0 3907264 671708 2794516 0 0 0 6526 1024 2015 67 9 9 16 0
6 0 0 3892040 673212 2842900 0 0 0 5431 491 1447 64 13 8 15 0
9 0 0 3882072 674228 2880744 0 0 0 8311 377 948 64 9 14 14 0
5 2 0 3828580 678084 2936000 0 0 0 6148 416 722 47 13 19 21 0
3 2 0 3764872 679672 2964268 0 0 0 5866 386 909 49 16 12 23 0
3 1 0 3705840 683140 3029056 0 0 34 8627 304 640 47 10 26 17 0
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
5 1 0 3665020 684576 3064740 0 0 0 7870 275 1132 61 13 16 10 0
6 0 0 3620108 686128 3104852 0 0 0 5588 274 848 58 13 19 10 0
7 0 0 3556256 687700 3162420 0 0 0 8943 292 487 47 9 24 20 0
5 0 0 3535932 689744 3225108 0 0 1 9206 359 853 62 13 10 15 0
3 1 0 3430196 690792 3254216 0 0 0 5650 171 647 61 20 15 4 0
1 1 0 3406220 693968 3291480 0 0 0 5216 327 390 45 9 21 25 0
5 1 0 3390264 694292 3304472 0 0 0 4809 254 658 61 8 15 16 0
1 2 0 3261136 699748 3378328 0 0 0 9194 1703 3241 57 13 21 9 0
6 1 0 3224132 701144 3403520 0 0 0 5246 337 576 55 8 10 27 0
3 1 0 3212252 703056 3446324 0 0 15 6206 240 798 73 13 10 5 0
4 1 0 3197564 704676 3465992 0 0 0 4244 339 585 65 8 8 18 0
5 3 0 3150860 706208 3543040 0 0 0 9143 320 800 65 13 11 11 0
6 0 0 3100700 706840 3562112 0 0 0 3948 312 923 69 14 6 10 0
6 0 0 3043272 707752 3625620 0 0 0 8512 330 676 67 10 7 16 0
5 0 0 3020464 708184 3648092 0 0 0 4369 331 1275 64 11 8 17 0
7 0 0 2901664 711148 3692596 0 0 0 6477 288 1020 78 13 5 4 0
7 0 0 2909912 712204 3738048 0 0 0 3596 320 514 80 6 2 13 0
6 0 0 2823392 712292 3773040 0 0 1 8041 234 556 79 16 2 2 0
5 0 0 2813468 712476 3788940 0 0 0 2408 215 408 79 5 5 11 0
6 0 0 2823888 712580 3801740 0 0 0 2435 2577 2917 81 8 4 7 0
6 2 0 2815312 712604 3805240 0 0 0 2678 188 456 71 16 8 4 0
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
6 0 0 2803880 712628 3809752 0 0 0 1141 168 412 87 5 1 7 0
5 0 0 2747044 713268 3844956 0 0 0 5545 1660 3907 72 10 7 11 0
7 0 0 2724024 713308 3845820 0 0 0 1165 294 1228 77 12 7 4 0
7 0 0 2753292 713356 3846460 0 0 0 936 242 909 85 9 3 4 0
2 4 0 2781840 713392 3848944 0 0 0 1380 351 1188 73 12 6 10 0
6 0 0 2764904 713424 3849464 0 0 0 406 254 1160 81 11 5 3 0
6 0 0 2750232 713464 3850372 0 0 0 1400 337 1060 77 13 4 6 0
4 0 0 3242124 713464 3375868 0 0 0 1352 304 743 68 18 7 6 0
5 0 0 3210540 713484 3379908 0 0 0 1621 713 1667 77 11 3 10 0
5 0 0 3230844 713552 3400784 0 0 0 721 379 1874 79 16 5 0 0
7 0 0 3170040 714612 3425764 0 0 0 5652 532 1541 57 13 13 17 0
8 0 0 3177324 715520 3451264 0 0 0 4188 477 1852 65 22 11 2 0
6 0 0 3154648 715592 3461440 0 0 12 4090 370 2125 70 11 7 12 0
7 2 0 3155840 715628 3453288 0 0 0 4634 283 832 77 11 7 5 0
5 0 0 3063224 716436 3497000 0 0 0 5470 661 2002 74 14 7 5 0
2 1 0 3328280 716524 3311632 0 0 0 3656 626 1604 61 15 8 17 0
1 1 0 3592780 716576 3013692 0 0 0 4708 559 1442 64 20 12 5 0
7 0 0 3530808 718712 3039400 0 0 0 4010 575 1256 61 11 14 14 0
8 0 0 3454940 720548 3067656 0 0 0 3148 545 1317 59 14 16 11 0
6 0 0 3529868 721480 3107216 0 0 0 4972 354 1233 47 11 30 12 0
15 0 0 3495260 722744 3136836 0 0 2 6173 515 1734 56 21 8 15 0
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 1 0 3454148 723992 3160840 0 0 5 4756 347 1590 47 19 15 19 0
6 1 0 3414760 724996 3222176 0 0 0 8916 354 1350 65 15 13 7 0
4 1 0 3378252 725508 3265320 0 0 0 7074 492 3057 56 14 14 16 0
4 0 0 3353492 726692 3291232 0 0 0 4771 341 1657 62 19 14 5 0
2 1 0 3325640 726976 3307088 0 0 0 5842 400 1106 51 11 16 22 0
5 0 0 3221244 730284 3353696 0 0 63 5809 381 786 58 13 10 19 0
5 0 0 3241588 731768 3389132 0 0 0 5178 352 898 59 11 9 20 0
3 1 0 3139184 734544 3430620 0 0 0 4730 260 1592 64 17 17 3 0
5 0 0 3087028 735380 3459036 0 0 0 4708 350 926 51 12 20 17 0
1 1 0 3100012 737632 3506632 0 0 0 6953 327 1702 55 20 18 7 0
1 1 0 2993264 738864 3572892 0 0 0 7829 242 1429 67 15 12 6 0
1 1 0 2893440 740268 3635704 0 0 12 8215 391 998 49 12 22 17 0
3 3 0 2818100 741556 3670112 0 0 272 4958 361 1450 70 16 10 3 0
5 0 0 2782916 742268 3682148 0 0 0 2568 379 1513 54 25 12 9 0
5 1 0 2900024 742472 3689300 0 0 0 3663 338 785 57 10 10 23 0
7 0 0 2895124 744224 3730860 0 0 0 4342 1441 2959 63 16 12 9 0
5 0 0 2853072 745136 3745572 0 0 0 5545 580 1108 42 10 26 22 0
4 0 0 2800308 746304 3758752 0 0 0 2893 367 1063 53 11 13 23 0
6 0 0 2762444 746560 3760404 0 0 0 2794 390 1062 70 11 8 11 0
7 1 0 3288240 746768 3348092 0 0 9 4522 476 1577 63 28 8 1 0
6 0 0 3227632 746996 3372568 0 0 0 6144 626 2074 51 16 17 16 0
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
3 2 0 3164676 747660 3403032 0 0 0 4810 514 1313 58 13 10 20 0
1 5 0 3161416 748372 3434648 0 0 0 4931 470 1948 51 17 21 11 0
5 0 0 3145284 748500 3445004 0 0 0 2307 382 4734 35 16 22 27 0
1 4 0 3134408 748528 3450084 0 0 0 3536 212 6309 52 26 16 7 0
4 2 0 3142704 748568 3466876 0 0 0 3669 342 4596 58 13 11 18 0
4 0 0 3087936 748796 3496352 0 0 0 5476 1699 7018 35 13 34 18 0
1 5 0 3012732 749608 3511964 0 0 0 4274 302 6475 54 17 20 9 0
0 2 0 3031792 752004 3552032 0 0 0 4940 358 4226 42 15 28 16 0
3 2 0 3009668 752568 3560248 0 0 0 2615 419 1694 48 15 17 20 0
0 1 0 2979776 755504 3593816 0 0 0 4196 418 1940 59 17 13 11 0
5 0 0 3106656 755744 3467372 0 0 0 2595 398 1833 44 20 18 18 0
1 5 0 3445968 756136 3147784 0 0 0 2899 377 2060 40 25 26 9 0
1 1 0 3387900 756996 3215964 0 0 0 3921 1496 3881 50 20 20 9 0
0 5 0 3317200 757236 3259132 0 0 0 9359 1984 3312 24 6 50 21 0
6 0 0 3287500 757516 3273288 0 0 0 2831 500 1302 48 9 24 19 0
4 1 0 3317400 758532 3295300 0 0 0 3979 280 1528 37 11 31 21 0
11 3 0 3284816 758776 3310512 0 0 0 4748 6138 57699 51 21 18 10 0
20 0 0 3215328 759388 3335228 0 0 0 2984 16304 146397 52 31 12 5 0
10 0 0 3137744 761108 3364864 0 0 0 3570 14833 100599 54 29 11 6 0
3 4 0 3185936 762204 3383328 0 0 0 4665 5853 39923 61 16 17 6 0
3 1 0 3197960 763204 3397152 0 0 0 4304 7602 41247 61 17 18 4 0
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
5 1 0 3156060 764436 3447752 0 0 0 6897 8428 55444 63 19 9 10 0
8 0 0 3056252 764796 3502828 0 0 0 8040 9656 66314 67 20 8 5 0
3 6 0 3037412 767668 3565072 0 0 0 8298 4995 45531 57 20 8 15 0
10 0 0 2932320 769380 3583160 0 0 0 3193 371 1008 34 35 22 9 0
BUG: soft lockup detected on CPU#1!
Call Trace:
[C000000087713610] [C00000000000EBDC] .show_stack+0x68/0x1b0 (unreliable)
[C0000000877136B0] [C000000000081230] .softlockup_tick+0xf0/0x128
[C000000087713760] [C00000000005961C] .run_local_timers+0x1c/0x30
[C0000000877137E0] [C0000000000202BC] .timer_interrupt+0xa8/0x474
[C0000000877138C0] [C0000000000034DC] decrementer_common+0xdc/0x100
--- Exception: 901 at .handle_futex_death+0x28/0x98
LR = .compat_exit_robust_list+0x108/0x1ac
[C000000087713BB0] [C0000000FAB61810] 0xc0000000fab61810 (unreliable)
[C000000087713C30] [C00000000006DA60] .compat_exit_robust_list+0x108/0x1ac
[C000000087713D00] [C000000000050C6C] .do_exit+0x1f8/0x980
[C000000087713DB0] [C0000000000514C0] .complete_and_exit+0x0/0x2c
[C000000087713E30] [C00000000000861C] syscall_exit+0x0/0x40
9 2 0 2939520 771364 3602556 0 0 0 3320 267 478 30 39 24 7 0
7 1 0 2865436 771984 3619916 0 0 0 2995 198 536 32 40 23 5 0
12 0 0 2841640 772456 3634672 0 0 0 2596 193 447 36 36 22 6 0
BUG: soft lockup detected on CPU#3!
Call Trace:
[C000000131003610] [C00000000000EBDC] .show_stack+0x68/0x1b0 (unreliable)
[C0000001310036B0] [C000000000081230] .softlockup_tick+0xf0/0x128
[C000000131003760] [C00000000005961C] .run_local_timers+0x1c/0x30
[C0000001310037E0] [C0000000000202BC] .timer_interrupt+0xa8/0x474
[C0000001310038C0] [C0000000000034DC] decrementer_common+0xdc/0x100
--- Exception: 901 at .handle_futex_death+0x30/0x98
LR = .compat_exit_robust_list+0x108/0x1ac
[C000000131003BB0] [C000000136FBF810] 0xc000000136fbf810 (unreliable)
[C000000131003C30] [C00000000006DA60] .compat_exit_robust_list+0x108/0x1ac
[C000000131003D00] [C000000000050C6C] .do_exit+0x1f8/0x980
[C000000131003DB0] [C0000000000514C0] .complete_and_exit+0x0/0x2c
[C000000131003E30] [C00000000000861C] syscall_exit+0x0/0x40
10 0 0 2891040 772456 3630992 0 0 0 777 64 177 21 55 24 0 0
10 0 0 2891940 772456 3630792 0 0 0 177 33 172 0 50 50 0 0
10 0 0 2892560 772456 3630792 0 0 0 160 28 168 0 50 50 0 0
25 0 0 2891464 772456 3630864 0 0 0 164 53 103 1 64 34 0 0
BUG: soft lockup detected on CPU#0!
Call Trace:
[C0000000A4307610] [C00000000000EBDC] .show_stack+0x68/0x1b0 (unreliable)
[C0000000A43076B0] [C000000000081230] .softlockup_tick+0xf0/0x128
[C0000000A4307760] [C00000000005961C] .run_local_timers+0x1c/0x30
[C0000000A43077E0] [C0000000000202BC] .timer_interrupt+0xa8/0x474
[C0000000A43078C0] [C0000000000034DC] decrementer_common+0xdc/0x100
--- Exception: 901 at .handle_futex_death+0x5c/0x98
LR = .compat_exit_robust_list+0x108/0x1ac
[C0000000A4307BB0] [C0000000D62E6810] 0xc0000000d62e6810 (unreliable)
[C0000000A4307C30] [C00000000006DA60] .compat_exit_robust_list+0x108/0x1ac
[C0000000A4307D00] [C000000000050C6C] .do_exit+0x1f8/0x980
[C0000000A4307DB0] [C0000000000514C0] .complete_and_exit+0x0/0x2c
[C0000000A4307E30] [C00000000000861C] syscall_exit+0x0/0x40
25 0 0 2891464 772456 3630868 0 0 0 101 16 30 0 77 23 0 0
26 0 0 2891472 772456 3630868 0 0 0 0 14 27 0 75 25 0 0
28 0 0 2891440 772456 3630868 0 0 0 0 16 28 0 75 25 0 0
28 0 0 2891440 772456 3630868 0 0 0 5 14 29 0 75 25 0 0
29 0 0 2891424 772456 3630868 0 0 0 0 20 28 0 75 25 0 0
29 0 0 2891388 772456 3630868 0 0 0 20 20 35 0 75 25 0 0
29 0 0 2891264 772456 3630868 0 0 0 10 32 30 0 75 25 0 0
29 0 0 2891212 772456 3630868 0 0 0 20 18 31 0 75 25 0 0
29 0 0 2890544 772456 3630868 0 0 0 41 22 39 0 75 25 0 0
29 0 0 2890664 772456 3630868 0 0 0 0 14 26 0 75 25 0 0
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
29 0 0 2890632 772456 3630868 0 0 0 0 13 29 0 75 25 0 0
29 0 0 2890616 772456 3630868 0 0 0 0 14 28 0 75 25 0 0
29 0 0 2890616 772456 3630868 0 0 0 0 23 28 0 75 25 0 0
29 0 0 2890616 772456 3630868 0 0 0 0 11 27 0 75 25 0 0
29 0 0 2890580 772456 3630868 0 0 0 17 15 29 0 75 25 0 0
29 0 0 2889748 772456 3630868 0 0 0 42 23 42 0 75 25 0 0
29 0 0 2889984 772456 3630868 0 0 0 0 15 32 0 75 25 0 0
29 0 0 2889968 772456 3630868 0 0 0 0 15 32 0 75 25 0 0
29 0 0 2889968 772456 3630868 0 0 0 0 15 32 0 75 25 0 0
29 0 0 2890060 772456 3630868 0 0 0 0 22 31 0 75 25 0 0
29 0 0 2890060 772456 3630868 0 0 0 0 11 28 0 75 25 0 0
29 0 0 2890060 772456 3630868 0 0 0 0 14 27 0 75 25 0 0
29 0 0 2890060 772456 3630868 0 0 0 0 17 27 0 75 25 0 0
29 0 0 2890060 772456 3630868 0 0 0 0 11 28 0 75 25 0 0
29 0 0 2890044 772456 3630868 0 0 0 0 14 58 0 75 25 0 0
29 0 0 2890044 772456 3630868 0 0 0 2 13 107 0 75 25 0 0
29 0 0 2890044 772456 3630868 0 0 0 0 24 105 0 75 25 0 0
29 0 0 2890012 772456 3630868 0 0 0 0 13 107 0 75 25 0 0
29 0 0 2890012 772456 3630868 0 0 0 0 14 105 0 75 25 0 0
29 0 0 2890012 772456 3630868 0 0 0 0 19 105 0 75 25 0 0
29 0 0 2890012 772456 3630868 0 0 0 0 14 105 0 75 25 0 0
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
29 0 0 2889996 772456 3630868 0 0 0 0 14 106 0 75 25 0 0
29 0 0 2889996 772456 3630868 0 0 0 0 26 106 0 75 25 0 0
29 0 0 2889996 772456 3630868 0 0 0 0 12 105 0 75 25 0 0
31 0 0 2789336 772560 3633644 0 0 0 2 17 107 0 75 25 0 0
31 0 0 2789336 772560 3633644 0 0 0 0 17 105 0 75 25 0 0
31 0 0 2789336 772560 3633644 0 0 0 0 16 106 0 75 25 0 0
31 0 0 2789332 772560 3633644 0 0 0 12 13 105 0 75 25 0 0
31 0 0 2789316 772560 3633644 0 0 0 0 22 107 0 75 25 0 0
31 0 0 2789316 772560 3633644 0 0 0 0 16 105 0 75 25 0 0
31 0 0 2789284 772560 3633644 0 0 0 0 16 107 0 75 25 0 0
31 0 0 2789284 772560 3633644 0 0 0 0 15 106 0 75 25 0 0
31 0 0 2789284 772560 3633644 0 0 0 0 16 105 0 75 25 0 0
31 0 0 2789284 772560 3633644 0 0 0 0 15 105 0 75 25 0 0
31 0 0 2789284 772560 3633644 0 0 0 2 13 107 0 75 25 0 0
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
31 0 0 2789268 772560 3633644 0 0 0 0 25 110 0 75 25 0 0
31 0 0 2789268 772560 3633644 0 0 0 0 14 108 0 75 25 0 0
31 0 0 2789268 772560 3633644 0 0 0 0 15 109 0 75 25 0 0
31 0 0 2789236 772560 3633644 0 0 0 0 18 110 0 75 25 0 0
31 0 0 2789236 772560 3633644 0 0 0 0 13 105 0 75 25 0 0
31 0 0 2789236 772560 3633644 0 0 0 0 16 105 0 75 25 0 0
31 0 0 2789236 772560 3633644 0 0 0 0 16 105 0 75 25 0 0
31 0 0 2789220 772560 3633644 0 0 0 0 29 107 0 75 25 0 0
31 0 0 2789220 772560 3633644 0 0 0 0 16 104 0 75 25 0 0
31 0 0 2789220 772560 3633644 0 0 0 2 20 107 0 75 25 0 0
31 0 0 2789220 772560 3633644 0 0 0 0 19 105 0 75 25 0 0
31 0 0 2789204 772560 3633644 0 0 0 0 11 106 0 75 25 0 0
31 0 0 2789188 772560 3633644 0 0 0 0 16 105 0 75 25 0 0
31 0 0 2789172 772560 3633644 0 0 0 0 22 106 0 75 25 0 0
31 0 0 2788772 772560 3633644 0 0 0 33 25 116 0 75 25 0 0
31 0 0 2788756 772560 3633644 0 0 0 6 18 107 0 75 25 0 0
31 0 0 2788756 772560 3633644 0 0 0 0 15 106 0 75 25 0 0
31 0 0 2788756 772560 3633644 0 0 0 0 17 105 0 75 25 0 0
31 0 0 2788756 772560 3633644 0 0 0 0 12 105 0 75 25 0 0
31 0 0 2788756 772560 3633644 0 0 0 2 13 107 0 75 25 0 0
31 0 0 2788708 772560 3633644 0 0 0 0 27 111 0 75 25 0 0
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/