Re: Crashing recent 6-STABLE, HP DL145G2 - LOR

Hi,

On 13.12.2005, at 19.21, Räihä Marko wrote: 
I have installed i386 6.0-RELEASE to two HP DL145G2-servers and they
crash soon after (1-2 minutes) booting up to login prompt. Sometimes
they boot up without crashing and sometimes with crashing.

I have now upgraded to RELENG_6 and crashing continues. What amazes
me is that these two machines do the same so I don't believe it would
be broken hardware.

I have copied dmesg, uname -a and kgdb backtrace information to
the following file on the web-site.

	http://www.zendata.fi/FreeBSD/debug_information.txt.gz

I have core file so I can dig more information if it is needed. 

I have now compiled new kernel with INVARIANTS and WITNESS. Reason for
crashing seems to be lock order reversal in ipf-module.

lock order reversal: (sleepable after non-sleepable)
1st 0xc4e42b40 ipf IP state rwlock (ipf IP state rwlock) @ /usr/src/ sys/modules/ipfilter/../../contrib/ipfilter/netinet/ip_state.c:2808
2nd 0xc0911744 user map (user map) @ /usr/src/sys/vm/vm_map.c:2997

How this should be fixed?


rigel# kgdb kernel.debug.6 vmcore.6
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:
lock order reversal: (sleepable after non-sleepable)
1st 0xc4e42b40 ipf IP state rwlock (ipf IP state rwlock) @ /usr/src/sys/modules/ipfilter/../../contrib/ipfilter/netinet/ip_state.c:2808
2nd 0xc0911744 user map (user map) @ /usr/src/sys/vm/vm_map.c:2997


Fatal trap 12: page fault while in kernel mode
fault virtual address = 0x88
fault code = supervisor read, page not present
instruction pointer = 0x20:0xc4e38dbf
stack pointer = 0x28:0xe5000c6c
frame pointer = 0x28:0xe5000c74
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 48 (swi4: clock)
trap number = 12
panic: page fault
Uptime: 56s
Dumping 1022 MB (2 chunks)
chunk 0: 1MB (151 pages) ... ok
chunk 1: 1023MB (261664 pages) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15

#0 doadump () at pcpu.h:165
165 pcpu.h: No such file or directory.
in pcpu.h
(kgdb)

================================================================================

(kgdb) where
#0 doadump () at pcpu.h:165
#1 0xc0630754 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:399
#2 0xc06309d2 in panic (fmt=0xc0832f51 "%s")
at /usr/src/sys/kern/kern_shutdown.c:555
#3 0xc07e96c8 in trap_fatal (frame=0xe5000c2c, eva=136)
at /usr/src/sys/i386/i386/trap.c:836
#4 0xc07e942f in trap_pfault (frame=0xe5000c2c, usermode=0, eva=136)
at /usr/src/sys/i386/i386/trap.c:744
#5 0xc07e90c1 in trap (frame=
{tf_fs = 8, tf_es = 40, tf_ds = 40, tf_edi = 0, tf_esi = -992533416, tf_ebp = -452981644, tf_isp = -452981672, tf_ebx = -991469056, tf_edx = -995095936, tf_ecx = 4, tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -991720001, tf_cs = 32, tf_eflags = 590406, tf_esp = -992533504, tf_ss = -992533504})
at /usr/src/sys/i386/i386/trap.c:434
#6 0xc07d8eba in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7 0xc4e38dbf in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) 


Regards,--
Marko Räihä

_______________________________________________
freebsd-stable@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: 6.2-STABLE (i386) Repeating crash (supervisor read, page not present)
    ... GDB is free software, covered by the GNU General Public License, and you are ... page fault while in kernel mode ... #5 0xc06b3f19 in trap (frame= ...
    (freebsd-stable)
  • Re: 100% repeatable crashes on 6.2-RELEASE-p3 (bt full)
    ... klon# kgdb kernel.debug /var/crash/vmcore.0 ... Fatal trap 12: page fault while in kernel mode ... frame pointer = 0x28:0xcc76fa78 ...
    (freebsd-stable)
  • Re: panic logging out on serial console
    ... page fault while in kernel mode ... help a lot unless someone tells me how I can skip the frame with ... the null pointer in kgdb. ...
    (freebsd-stable)
  • Panic in ip_input
    ... of data from my laptop to the desktop system: ... page fault while in kernel mode ... #14 0xc06d9ae0 in syscall (frame= ...
    (freebsd-current)
  • [Fwd: Re: 6-STABLE filesystem related panics/locks (kgdb output)]
    ... On one of our NFS servers, we've seen repeated filesystem issues with two of the filesystems. ... This morning, prior to rebooting the system to get it out of this state, I began unmounting filesystems in case of a panic, and after unmounting two of the filesystems, I tried unmounting the third, and a panic ensued. ... Here's some kgdb output from when the system was wedged on /scr01, but the unmount of /scr02 caused a panic: ... #13 0xc0896c0b in syscall (frame= ...
    (freebsd-current)