Bushel of Oopses 2.2.5-ac6 w/ Ksymoops

Jordan Mendelson (jordy@wserv.com)
Mon, 12 Apr 1999 01:58:15 -0400 (EDT)


I've been trying to track down a problem with one of our SMP boxes for
some time now. Someone decided to run the Mersene Prime searching program
and the box crashed a little over 7 hours later.

I decided to hookup serial console and try it again and see if I could get
some oopses. Note that none of these were fatal, the machine kept
running.. however a number of programs were stuck in the "D" state and the
load was over 9... ps didn't show any of the functions they were stuck
in... but just a number unfortunately...

This machine is a dual P2 266 with 256 megs of RAM, an Intel EEpro 100 and
a Buslogic SCSI controller handling 2 SCSI-2 drives running 2.2.5-ac6
however it has been unstable throughout the 2.2.x series of kernels but
was stable in the 2.1.x series.

-- Versions installed: (if some fields are empty or looks
-- unusual then possibly you have very old versions)
Linux solid 2.2.5-ac6 #3 SMP Sun Apr 11 03:02:08 EDT 1999 i686 unknown
Kernel modules 2.1.85
Gnu C egcs-2.91.66
Binutils 2.8.1.0.23
Linux C Library 5.4.44
Dynamic linker ldd: version 1.9.9
Procps 1.2.7
Mount 2.9f
Net-tools (1999-01-31)
Kbd command
Sh-utils 1.16
Modules Loaded

Unable to handle kernel NULL pointer dereference at virtual address 00000018
current->tss.cr3 = 0febc000, %cr3 = 0febc000
*pde = 00000000
Oops: 0000
CPU: 1
EIP: 0010:[<c011e913>]
EFLAGS: 00010202
eax: 00001240 ebx: 000e9000 ecx: c0229160 edx: 00000010
esi: c2f65ba0 edi: c99cf000 ebp: 000ea000 esp: cf26df18
ds: 0018 es: 0018 ss: 0018
Process squid (pid: 400, process nr: 20, stackpage=cf26d000)
Stack: c03e0380 000ea000 c0229160 c011ede2 c20ad8a0 000ea000 00000000 00001000
0c9385d8 00000000 cf26c000 c022915c 00001000 000e9000 00003000 00012000
00000000 00000001 00000001 000e9000 c2f65ba0 c011f197 c20ad8a0 c20ad8b4
Call Trace: [<c011ede2>] [<c011f197>] [<c011f0e4>] [<c01273aa>] [<c0108be8>]
Code: 39 72 08 75 f4 39 6a 0c 75 ef f0 ff 42 14 b8 02 00 00 00 f0

>>EIP; c011e913 <try_to_read_ahead+7b/11c>
Trace; c011ede2 <do_generic_file_read+2fa/5fc>
Trace; c011f197 <generic_file_read+63/7c>
Trace; c011f0e4 <file_read_actor+0/50>
Trace; c01273aa <sys_read+c2/e8>
Trace; c0108be8 <system_call+34/38>
Code; c011e913 <try_to_read_ahead+7b/11c> 00000000 <_EIP>:
Code; c011e913 <try_to_read_ahead+7b/11c> 0: 39 72 08 cmpl %esi,0x8(%edx)
Code; c011e916 <try_to_read_ahead+7e/11c> 3: 75 f4 jne fffffff9 <_EIP+0xfffffff9> c011e90c <try_to_read_ahead+74/11c>
Code; c011e918 <try_to_read_ahead+80/11c> 5: 39 6a 0c cmpl %ebp,0xc(%edx)
Code; c011e91b <try_to_read_ahead+83/11c> 8: 75 ef jne fffffff9 <_EIP+0xfffffff9> c011e90c <try_to_read_ahead+74/11c>
Code; c011e91d <try_to_read_ahead+85/11c> a: f0 ff 42 14 lock incl 0x14(%edx)
Code; c011e921 <try_to_read_ahead+89/11c> e: b8 02 00 00 00 movl $0x2,%eax
Code; c011e926 <try_to_read_ahead+8e/11c> 13: f0 00 00 lock addb %al,(%eax)

Unable to handle kernel NULL pointer dereference at virtual address 00000018
current->tss.cr3 = 00f49000, %cr3 = 00f49000
*pde = 00000000
Oops: 0000
CPU: 1
EIP: 0010:[<c011e913>]
EFLAGS: 00010202
eax: 00001240 ebx: 0010e000 ecx: c0229160 edx: 00000010
esi: ccd69cb0 edi: cbb6f000 ebp: 00118000 esp: cf135f18
ds: 0018 es: 0018 ss: 0018
Process cat (pid: 4373, process nr: 69, stackpage=cf135000)
Stack: c03920b8 00118000 c0229160 c011ede2 c20ad9c0 00118000 00000000 00001000
0804dc88 00000000 cf134000 c02290f0 0000a000 000fc000 00013000 00012000
00000000 00000002 00000001 000fc000 ccd69cb0 c011f197 c20ad9c0 c20ad9d4
Call Trace: [<c011ede2>] [<c011f197>] [<c011f0e4>] [<c01273aa>] [<c0108be8>]
Code: 39 72 08 75 f4 39 6a 0c 75 ef f0 ff 42 14 b8 02 00 00 00 f0

>>EIP; c011e913 <try_to_read_ahead+7b/11c>
Trace; c011ede2 <do_generic_file_read+2fa/5fc>
Trace; c011f197 <generic_file_read+63/7c>
Trace; c011f0e4 <file_read_actor+0/50>
Trace; c01273aa <sys_read+c2/e8>
Trace; c0108be8 <system_call+34/38>
Code; c011e913 <try_to_read_ahead+7b/11c> 00000000 <_EIP>:
Code; c011e913 <try_to_read_ahead+7b/11c> 0: 39 72 08 cmpl %esi,0x8(%edx)
Code; c011e916 <try_to_read_ahead+7e/11c> 3: 75 f4 jne fffffff9 <_EIP+0xfffffff9> c011e90c <try_to_read_ahead+74/11c>
Code; c011e918 <try_to_read_ahead+80/11c> 5: 39 6a 0c cmpl %ebp,0xc(%edx)
Code; c011e91b <try_to_read_ahead+83/11c> 8: 75 ef jne fffffff9 <_EIP+0xfffffff9> c011e90c <try_to_read_ahead+74/11c>
Code; c011e91d <try_to_read_ahead+85/11c> a: f0 ff 42 14 lock incl 0x14(%edx)
Code; c011e921 <try_to_read_ahead+89/11c> e: b8 02 00 00 00 movl $0x2,%eax
Code; c011e926 <try_to_read_ahead+8e/11c> 13: f0 00 00 lock addb %al,(%eax)

Unable to handle kernel NULL pointer dereference at virtual address 00000018
current->tss.cr3 = 01672000, %cr3 = 01672000
*pde = 00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c011e7e2>]
EFLAGS: 00010202
eax: c03df908 ebx: c8873760 ecx: 00000010 edx: 0001910e
esi: 00000000 edi: 40009000 ebp: 00000400 esp: c8d6bea0
ds: 0018 es: 0018 ss: 0018
Process webalizer (pid: 4519, process nr: 30, stackpage=c8d6b000)
Stack: 40009000 c8873760 00000010 0c887376 c013e594 c8873760 0000c000 c7cffc00
00000400 c65c1560 ffffffea c88737ac c8d6a000 c632e080 00000400 0000c000
00000000 c8d6bf08 00000212 c4090000 00000000 00000000 c0fd6a00 00000000
Call Trace: [<c013e594>] [<c01249e5>] [<c011c34a>] [<c011dca9>] [<c011df52>] [<c01274c1>] [<c013e2e8>]
[<c0108be8>]
Code: 39 59 08 75 e1 8b 5c 24 20 39 59 0c 75 d8 f0 ff 41 14 b8 02

>>EIP; c011e7e2 <update_vm_cache+7a/130>
Trace; c013e594 <ext2_file_write+2ac/4f8>
Trace; c01249e5 <free_page_and_swap_cache+5d/60>
Trace; c011c34a <zap_page_range+13e/1bc>
Trace; c011dca9 <unmap_fixup+109/118>
Trace; c011df52 <do_munmap+212/228>
Trace; c01274c1 <sys_write+f1/124>
Trace; c013e2e8 <ext2_file_write+0/4f8>
Trace; c0108be8 <system_call+34/38>
Code; c011e7e2 <update_vm_cache+7a/130> 00000000 <_EIP>:
Code; c011e7e2 <update_vm_cache+7a/130> 0: 39 59 08 cmpl %ebx,0x8(%ecx)
Code; c011e7e5 <update_vm_cache+7d/130> 3: 75 e1 jne ffffffe6 <_EIP+0xffffffe6> c011e7c8 <update_vm_cache+60/130>
Code; c011e7e7 <update_vm_cache+7f/130> 5: 8b 5c 24 20 movl 0x20(%esp,1),%ebx
Code; c011e7eb <update_vm_cache+83/130> 9: 39 59 0c cmpl %ebx,0xc(%ecx)
Code; c011e7ee <update_vm_cache+86/130> c: 75 d8 jne ffffffe6 <_EIP+0xffffffe6> c011e7c8 <update_vm_cache+60/130>
Code; c011e7f0 <update_vm_cache+88/130> e: f0 ff 41 14 lock incl 0x14(%ecx)
Code; c011e7f4 <update_vm_cache+8c/130> 12: b8 02 00 00 00 movl $0x2,%eax

Unable to handle kernel NULL pointer dereference at virtual address 00000018
current->tss.cr3 = 05b89000, %cr3 = 05b89000
*pde = 00000000
Oops: 0000
CPU: 1
EIP: 0010:[<c011ec4b>]
EFLAGS: 00010202
eax: 00001240 ebx: 00027000 ecx: c0229160 edx: 00000010
esi: ccc62dd0 edi: 00000000 ebp: 00000000 esp: c8d6bf3c
ds: 0018 es: 0018 ss: 0018
Process bzip2 (pid: 4523, process nr: 30, stackpage=c8d6b000)
Stack: 00000000 c8d6a000 c0229160 00000000 00027000 00000000 00012000 00000000
00000000 00000000 00027000 ccc62dd0 c011f197 c20ad120 c20ad134 c8d6bf8c
c011f0e4 c20ad120 ffffffea 00000000 00000000 00001000 40009000 00000000
Call Trace: [<c011f197>] [<c011f0e4>] [<c01273aa>] [<c0108be8>]
Code: 39 72 08 75 f0 39 5a 0c 75 eb f0 ff 42 14 b8 02 00 00 00 f0

>>EIP; c011ec4b <do_generic_file_read+163/5fc>
Trace; c011f197 <generic_file_read+63/7c>
Trace; c011f0e4 <file_read_actor+0/50>
Trace; c01273aa <sys_read+c2/e8>
Trace; c0108be8 <system_call+34/38>
Code; c011ec4b <do_generic_file_read+163/5fc> 00000000 <_EIP>:
Code; c011ec4b <do_generic_file_read+163/5fc> 0: 39 72 08 cmpl %esi,0x8(%edx)
Code; c011ec4e <do_generic_file_read+166/5fc> 3: 75 f0 jne fffffff5 <_EIP+0xfffffff5> c011ec40 <do_generic_file_read+158/5fc>
Code; c011ec50 <do_generic_file_read+168/5fc> 5: 39 5a 0c cmpl %ebx,0xc(%edx)
Code; c011ec53 <do_generic_file_read+16b/5fc> 8: 75 eb jne fffffff5 <_EIP+0xfffffff5> c011ec40 <do_generic_file_read+158/5fc>
Code; c011ec55 <do_generic_file_read+16d/5fc> a: f0 ff 42 14 lock incl 0x14(%edx)
Code; c011ec59 <do_generic_file_read+171/5fc> e: b8 02 00 00 00 movl $0x2,%eax
Code; c011ec5e <do_generic_file_read+176/5fc> 13: f0 00 00 lock addb %al,(%eax)

Unable to handle kernel NULL pointer dereference at virtual address 00000018
current->tss.cr3 = 0a6ea000, %cr3 = 0a6ea000
*pde = 00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c011e7e2>]
EFLAGS: 00010202
eax: c03df908 ebx: c2120110 ecx: 00000010 edx: 00018424
esi: 00000000 edi: bffff2fa ebp: 00000006 esp: ca755ea0
ds: 0018 es: 0018 ss: 0018
Process mprime (pid: 3708, process nr: 67, stackpage=ca755000)
Stack: bffff2fa c2120110 00000010 0c212011 c013e594 c2120110 0005b000 cffbd800
00000006 c65c1b60 ffffffea c212015c ca754000 c6a8a620 00000006 0005a006
00000000 ca755f08 00000000 00000212 00000000 00000000 c0fd6a00 00000ffa
Call Trace: [<c013e594>] [<c0171427>] [<c0171490>] [<c0163627>] [<c010df39>] [<c010f32e>] [<c01274c1>]
[<c013e2e8>] [<c0108be8>]
Code: 39 59 08 75 e1 8b 5c 24 20 39 59 0c 75 d8 f0 ff 41 14 b8 02

>>EIP; c011e7e2 <update_vm_cache+7a/130>
Trace; c013e594 <ext2_file_write+2ac/4f8>
Trace; c0171427 <raw_rcv_skb+a7/dc>
Trace; c0171490 <raw_rcv+34/3c>
Trace; c0163627 <ip_local_deliver+153/18c>
Trace; c010df39 <timer_interrupt+95/11c>
Trace; c010f32e <smp_local_timer_interrupt+ca/134>
Trace; c01274c1 <sys_write+f1/124>
Trace; c013e2e8 <ext2_file_write+0/4f8>
Trace; c0108be8 <system_call+34/38>
Code; c011e7e2 <update_vm_cache+7a/130> 00000000 <_EIP>:
Code; c011e7e2 <update_vm_cache+7a/130> 0: 39 59 08 cmpl %ebx,0x8(%ecx)
Code; c011e7e5 <update_vm_cache+7d/130> 3: 75 e1 jne ffffffe6 <_EIP+0xffffffe6> c011e7c8 <update_vm_cache+60/130>
Code; c011e7e7 <update_vm_cache+7f/130> 5: 8b 5c 24 20 movl 0x20(%esp,1),%ebx
Code; c011e7eb <update_vm_cache+83/130> 9: 39 59 0c cmpl %ebx,0xc(%ecx)
Code; c011e7ee <update_vm_cache+86/130> c: 75 d8 jne ffffffe6 <_EIP+0xffffffe6> c011e7c8 <update_vm_cache+60/130>
Code; c011e7f0 <update_vm_cache+88/130> e: f0 ff 41 14 lock incl 0x14(%ecx)
Code; c011e7f4 <update_vm_cache+8c/130> 12: b8 02 00 00 00 movl $0x2,%eax

Unable to handle kernel NULL pointer dereference at virtual address 00000018
current->tss.cr3 = 0aeed000, %cr3 = 0aeed000
*pde = 00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c011ec4b>]
EFLAGS: 00010202
eax: 00001240 ebx: 00000000 ecx: c0229160 edx: 00000010
esi: c295f650 edi: 00000000 ebp: 00000000 esp: ca6e9f3c
ds: 0018 es: 0018 ss: 0018
Process grep (pid: 4597, process nr: 72, stackpage=ca6e9000)
Stack: 00000000 ca6e8000 c0229160 00000000 00000000 cca8fe60 c7809d40 00000000
ca6e8000 00000001 00000000 c295f650 c011f197 cca8fe60 cca8fe74 ca6e9f8c
c011f0e4 cca8fe60 ffffffea 00000000 00000000 00008000 08061000 00000000
Call Trace: [<c011f197>] [<c011f0e4>] [<c01273aa>] [<c0108be8>]
Code: 39 72 08 75 f0 39 5a 0c 75 eb f0 ff 42 14 b8 02 00 00 00 f0

>>EIP; c011ec4b <do_generic_file_read+163/5fc>
Trace; c011f197 <generic_file_read+63/7c>
Trace; c011f0e4 <file_read_actor+0/50>
Trace; c01273aa <sys_read+c2/e8>
Trace; c0108be8 <system_call+34/38>
Code; c011ec4b <do_generic_file_read+163/5fc> 00000000 <_EIP>:
Code; c011ec4b <do_generic_file_read+163/5fc> 0: 39 72 08 cmpl %esi,0x8(%edx)
Code; c011ec4e <do_generic_file_read+166/5fc> 3: 75 f0 jne fffffff5 <_EIP+0xfffffff5> c011ec40 <do_generic_file_read+158/5fc>
Code; c011ec50 <do_generic_file_read+168/5fc> 5: 39 5a 0c cmpl %ebx,0xc(%edx)
Code; c011ec53 <do_generic_file_read+16b/5fc> 8: 75 eb jne fffffff5 <_EIP+0xfffffff5> c011ec40 <do_generic_file_read+158/5fc>
Code; c011ec55 <do_generic_file_read+16d/5fc> a: f0 ff 42 14 lock incl 0x14(%edx)
Code; c011ec59 <do_generic_file_read+171/5fc> e: b8 02 00 00 00 movl $0x2,%eax
Code; c011ec5e <do_generic_file_read+176/5fc> 13: f0 00 00 lock addb %al,(%eax)

If you need any additional information, I'd be happy to provide it.

Jordan

--
Jordan Mendelson     : http://jordy.wserv.com
Web Services, Inc.   : http://www.wserv.com

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/