RE: FreeBSD 4.11 P13 Crash

I do not want to jinx myself, but after back revving to FreeBSD 4.9 +
patches, the system has been up for nearly 19 days. It just seems odd that
when I moved to 4.11, it worked for maybe 14 days to 30 days without a
crash. Then the crashing would randomly occur more often until it got to a
point where it could not make it past the week without a reboot. I might
have upgrading IPFilter in that time frame, but I did not keep track of that
change unfortunately.

So, this is a strong indication of a software bug. If I had to guess, I
think it is related to IPFilter in conjunction with 4 Intel nics and/or
PPPoE. I run the same version of IPfilter on another box with only 2 nics
and it has 100 day+ uptimes. I do not mind staying with 4.9 though and I
suppose interest in resolving some ancient bug that occurs in such a
specific setup is pretty low. I'm just glad I can rely on my FreeBSD box
again!

I'll keep you guys posted if it makes it past the one month marker and
hopefully reach back to the old day days of 110+ day uptimes :) Thanks
again guys for your help!



- Carroll Kong

-----Original Message-----
From: owner-freebsd-hackers@xxxxxxxxxxx
[mailto:owner-freebsd-hackers@xxxxxxxxxxx] On Behalf Of Carroll Kong
Sent: Monday, March 06, 2006 12:36 PM
To: hackers@xxxxxxxxxxx
Subject: RE: FreeBSD 4.11 P13 Crash

Well bad news. It happened again even with a new CPU and new
PowerSupply.
However, the good news is that it seems to be saving the core
dumps a bit more consistently now. I swapped the motherboard
back to the old one.
Honestly, I've had similar core dumps in either case, I'm
starting to think it isn't the hardware but more of a
software or software configuration issue (one that works but
is apparently not stable). I can swap the motherboard back
to the new one, but I got the same error in either case.

It sure looks a lot like the other backtraces, but I suppose
if something corrupted the data in memory, it could still be anything.

So far I am thinking
- IPFilter intermittent bug with some packets, but I run a box with
112 days of uptime with the same version of IPFilter,
albeit not with
4 NICs.

It keeps failing around the net or ppp process. It might not
mean anything though since the box will always log 'junk'.
Although, I find it odd that it has never crashed when using
the other, cable link, ONLY on the PPP process. Only an
experienced hacker can tell me if my hypothesis is correct in
pointing the finger closer to PPP. Then again, there is no
'process' to refer to if the issue was on the cable link!
Maybe a bug in handling 4 FXP Intel cards? I already swapped
one of them (the other 2 are built onboard, and the last one
I have not swapped out yet that connects to the cable link).

- 3Ware driver is flakey, but I have a 4.10 box with 3Ware that is
somewhat stable
- CPU (I would tend to think this would result in HARD lock ups vs
Fatal Trap 12s though)

New CPU didn't fix it, so let's scratch that out. In fact,
it's been pulled from a working system.

- PowerSupply (I suppose anything is possible, please note
it is on an
APC UPS, but the power supply might be delivering bad juice?)

New Power supply. Antec 380 or so. I don't have a method of
testing it so it 'could' be a bad new power supply but
honestly, I would expect it to have crashed with a different error.

- Harddisks and 3Ware driver have incompatible firmware
issue, I doubt
this is it though since I purchased new Seagates in 9/2004 for the
RAID1, then I added another Seagate as a JBOD, and that disk is not
being written to during the crash.

This is still a possibility, although it seems to fail in a
memory operation. Unless the 3Ware somehow corrupted memory
ahead of time, which seems kind of odd but possible.

Someone suggested I go back to 4.9. I don't mind doing so
although I wonder if I would be vulnerable to certain
security issues. Furthermore, I'm not sure how to do this
right. I would guess cvsup with 4.9_RELEASE tag?

Anyway, I am going to try to go back to 4.9, but wanted to
throw some more information to the list to see if anyone had
any other ideas.

The only hardware I have not changed so far is
- cdrom
- floppy
- case

:)

Here is another backtrace. It seems to be doing the standard
log to the ipf.log file thing, then die during an mbuf operation.

Fatal trap 12: page fault while in kernel mode
fault virtual address = 0x41f59
fault code = supervisor read, page not present
instruction pointer = 0x8:0xc0192696
stack pointer = 0x10:0xd71cbbd0
frame pointer = 0x10:0xd71cbbd8
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 105 (ppp)
interrupt mask = net tty
trap number = 12
panic: page fault

---------------

#0 dumpsys () at ../../kern/kern_shutdown.c:487
487 if (dumping++) {
(kgdb) bt
#0 dumpsys () at ../../kern/kern_shutdown.c:487
#1 0xc0173f3f in boot (howto=256) at ../../kern/kern_shutdown.c:316
#2 0xc0174364 in poweroff_wait (junk=0xc02f2aac,
howto=-1070651985) at
../../kern/kern_shutdown.c:595
#3 0xc02a77ba in trap_fatal (frame=0xd71cbb90, eva=270169) at
../../i386/i386/trap.c:974
#4 0xc02a748d in trap_pfault (frame=0xd71cbb90, usermode=0,
eva=270169) at
../../i386/i386/trap.c:867
#5 0xc02a704b in trap (frame={tf_fs = -1072562160, tf_es =
16, tf_ds = -686030832, tf_edi = 6757530, tf_esi = -1055310592,
tf_ebp = -685982760, tf_isp = -685982788, tf_ebx =
270169, tf_edx = 6757530, tf_ecx = -1055840256, tf_eax = -28864,
tf_trapno = 12, tf_err = 0, tf_eip = -1072093546, tf_cs
= 8, tf_eflags = 66054, tf_esp = -1055310592, tf_ss = -1055310592})
at ../../i386/i386/trap.c:466
#6 0xc0192696 in m_tag_delete_chain (m=0xc1193d00, t=0x0) at
../../kern/uipc_mbuf2.c:358
#7 0xc01904d3 in m_free (m=0xc1193d00) at ../../kern/uipc_mbuf.c:734
#8 0xc0190606 in m_freem (m=0xc117ef00) at ../../kern/uipc_mbuf.c:763
#9 0xc0127df8 in fr_check (ip=0xc117ef30, hlen=20,
ifp=0xc21a9508, out=0,
mp=0xd71cbce8)
at ../../contrib/ipfilter/netinet/fil.c:1387
#10 0xc01d7d06 in ip_input (m=0xc117ef00) at
../../netinet/ip_input.c:478
#11 0xc01d838b in ipintr () at ../../netinet/ip_input.c:971
#12 0xc0299ee9 in swi_net_next ()
#13 0xc016e7c8 in lockmgr (lkp=0xc21a9600, flags=16973826,
interlkp=0xd71c162c, p=0xd48bd5a0) at ../../kern/kern_lock.c:355
#14 0xc019fda8 in vop_stdlock (ap=0xd71cbdd0) at
../../kern/vfs_default.c:256
#15 0xc025a9c9 in ufs_vnoperatespec (ap=0xd71cbdd0) at
../../ufs/ufs/ufs_vnops.c:2394
#16 0xc01a9ffd in vn_lock (vp=0xd71c15c0, flags=131074,
p=0xd48bd5a0) at
vnode_if.h:861
#17 0xc01ad842 in spec_write (ap=0xd71cbe64) at
../../miscfs/specfs/spec_vnops.c:284
#18 0xc025a3ac in ufsspec_write (ap=0xd71cbe64) at
../../ufs/ufs/ufs_vnops.c:1827
#19 0xc025a9c9 in ufs_vnoperatespec (ap=0xd71cbe64) at
../../ufs/ufs/ufs_vnops.c:2394
#20 0xc01a9b9a in vn_write (fp=0xc21a8c40, uio=0xd71cbed4,
cred=0xc219b780, flags=0, p=0xd48bd5a0) at vnode_if.h:363
#21 0xc018330d in dofilewrite (p=0xd48bd5a0, fp=0xc21a8c40,
fd=9, buf=0xbfbfe89c, nbyte=213, offset=-1, flags=0)
at ../../sys/file.h:163
#22 0xc01831c6 in write (p=0xd48bd5a0, uap=0xd71cbf80) at
../../kern/sys_generic.c:329
#23 0xc02a7a69 in syscall2 (frame={tf_fs = 134938671, tf_es =
47, tf_ds = -1078001617, tf_edi = 134996736, tf_esi = 213,
tf_ebp = -1077940064, tf_isp = -685981740, tf_ebx =
-1077942112, tf_edx = 0, tf_ecx = 13, tf_eax = 4, tf_trapno = 7,
tf_err = 2, tf_eip = 673683504, tf_cs = 31, tf_eflags =
663, tf_esp = -1077942172, tf_ss = 47})
at ../../i386/i386/trap.c:1175
#24 0xc0298a85 in Xint0x80_syscall ()
#25 0x80655de in ?? ()
#26 0x806c2fb in ?? ()
#27 0x806c21d in ?? ()
#28 0x807470a in ?? ()
#29 0x8083a78 in ?? ()
#30 0x805b84b in ?? ()
#31 0x804d484 in ?? ()
#32 0x806ed77 in ?? ()
#33 0x806e967 in ?? ()
#34 0x804b62a in ?? ()
(kgdb) quit
daemon# nm /kernel | grep c0192696
daemon# nm /kernel | grep c019269
daemon# nm /kernel | grep c01926
c01926e8 T m_tag_copy
c0192658 T m_tag_delete
c0192674 T m_tag_delete_chain
c0192600 T m_tag_free
c01926ac T m_tag_locate
c0192614 T m_tag_prepend
c0192628 T m_tag_unlink

-----Original Message-----
From: Carroll Kong
Sent: Monday, February 27, 2006 8:53 PM
To: 'hackers@xxxxxxxxxxx'
Subject: FreeBSD 4.11 P13 Crash

Okay this time my kernel was recompiled so there are no modules to
make it easier to see all of the symbols.

Sometimes the box cycles through the fatal traps 12. Other
times it
does not. Based on my other Fatal trap errors, it seems to
interrupt
more often with the m_tag_delete function.
I don't think this necessarily means the problem is with
IPFilter or
PPP mostly because this box acts as a firewall and logs
constantly.
Therefore, it is not surprising it always fails after logging with
IPFilter, but I am always open to the possibility.

This box was stable before I upgraded from 4.9->4.11. Among one of
the software changes was probably the change of IPFilter.
I used to
use the IPFilter 3.4.33pre modules, but after I moved to
4.11 I just
used the distribution packaged 3.4.35. This might be the source of
the problem, but I could not google for relevant entries.

I have since swapped the RAM, motherboard, RAM again (I
bought another
stick thinking maybe my new RAM was coincidentally bugged),
one of the
Intel NICs, and my 3Ware controller. The problem still
occurred and
actually more frequently. The usual frequency was about 14 days or
so. It just crashed in less than 23 hours and then again within 25
minutes.

The final pieces of hardware that still can be swapped is the other
Intel NIC (but this NIC is NOT connected to the PPPoE), CPU, Power
Supply, CDROM (not used), Harddisks, or Case. :)

I tried disabling physical swap completely, and the system still
crashed, so I doubt it is the 3Ware, but who knows.

So far I am thinking
- IPFilter intermittent bug with some packets, but I run a box with
112 days of uptime with the same version of IPFilter,
albeit not with
4 NICs.
- 3Ware driver is flakey, but I have a 4.10 box with 3Ware that is
somewhat stable
- CPU (I would tend to think this would result in HARD lock ups vs
Fatal Trap 12s though)
- PowerSupply (I suppose anything is possible, please note
it is on an
APC UPS, but the power supply might be delivering bad juice?)
- Harddisks and 3Ware driver have incompatible firmware
issue, I doubt
this is it though since I purchased new Seagates in 9/2004 for the
RAID1, then I added another Seagate as a JBOD, and that disk is not
being written to during the crash.

I am tempted to consider upgrading to 5.X, but I am a conservative
person and somehow doubt 4.X is the source of the problem as the
system worked fine for over a year.

The box does a lot of things however I omitted this information to
avoid flooding the list with too much information since it
has worked
fine for a year in the past.
As a note, the problem is NOT load related. In fact, one time the
fatal panic said the running process was "idle". :)
Furthermore, I
haven't really updated the software unnecessarily except
for security
issues and the system has been stable in the past with the same
hardware and same software. I am very conservative when it
comes to
servers, so this seems like a hardware issue but I already
swapped so
much of it, I am beginning to wonder.

I am going to buy a new CPU and power supply as I have
replaced nearly
every other part by now.

I have included my dmesg, nm greps for the functions, a backtrace,
uname output. I have the kernel dump so if there are any commands
someone needs me to punch through, I will gladly do so. I included
some of my own feeble debugging. I didn't like the line that said
"address is out of bounds" in one of the mbuf structures. I am
guessing that means the mbuf was already corrupted way
before we got
there. Any suggestions and advice are welcome. Thanks in advance!



Fatal trap 12: page fault while in kernel mode
fault virtual address = 0xc11e4402
fault code = supervisor write, page not present
instruction pointer = 0x8:0xc018ffcf
stack pointer = 0x10:0xc02fa6f0
frame pointer = 0x10:0xc02fa704
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = Idle
interrupt mask = net tty bio cam
trap number = 12
panic: page fault
Uptime: 6h5m10s
twe0: Cannot delete unit. error = 16

Fatal trap 12: page fault while in kernel mode
fault virtual address = 0xc11e4402
fault code = supervisor write, page not present
instruction pointer = 0x8:0xc018ffcf
stack pointer = 0x10:0xc02fa444
frame pointer = 0x10:0xc02fa458
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = Idle
interrupt mask = net tty bio cam
tx0, limit 0xfffff, type 0x1b

nm -n /kernel | grep c018f
c018f058 T accept_filt_del
c018f07c T accept_filt_get
c018f0b4 T accept_filt_generic_mod_event
c018f134 t net_init_domain
c018f1bc T net_add_domain
c018f1ec t domaininit
c018f244 T pffindtype
c018f290 T pffindproto
c018f304 T pfctlinput
c018f34c T pfctlinput2
c018f3a4 t pfslowtimo
c018f3fc t pffasttimo
c018f45c t tunable_mbinit
c018f4ac t mbinit
c018f53c T m_mballoc
c018f5f8 T m_mballoc_wait
c018f7e8 T m_clalloc
c018f8b4 T m_clalloc_wait
c018f9a0 T m_retry
c018fa74 T m_retryhdr
c018fb60 t m_reclaim
c018fbb0 T m_get
c018fc54 T m_gethdr
c018fd0c T m_getclr
c018fdd0 T m_getcl

-------------------------------------------------------------------

Fatal trap 12: page fault while in kernel mode
fault virtual address = 0x28067100
fault code = supervisor read, page not present
instruction pointer = 0x8:0xc0192696
stack pointer = 0x10:0xd71cbbd0
frame pointer = 0x10:0xd71cbbd8
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 110 (ppp)
interrupt mask = net tty
trap number = 12
panic: page fault

syncing disks... 7
done
Uptime: 25m51s
twe0: Cannot delete unit. error = 16

dumping to dev #twed/0x20001, offset 3146240 dump 511 510 509
508 507 506 505 504 503 502 501 500 499 498 497 496 495 494
493 492 491 490 489 488 487 486 485 484 483 482 481 480 479
478 477 476 475 474 473 472 471 470 469 468 467 466 465 464
463 462 461 460 459 458 457 456 455 454 453 452 451 450 449
448 447 446 445 444 443 442 441 440 439 438 437 436 435 434
433 432 431 430 429 428 427 426 425 424 423 422 421 420 419
418 417 416 415 414 413 412 411 410 409 408 407 406 405 404
403 402 401 400 399 398 397 396 395 394 393 392 391 390 389
388 387 386 385 384 383 382 381 380 379 378 377 376 375 374
373 372 371 370 369 368 367 366 365 364 363 362 361 360 359
358 357 356 355 354 353 352 351 350 349 348 347 346 345 344
343 342 341 340 339 338 337 336 335 334 333 332 331 330 329
328 327 326 325 324 323 322 321 320 319 318 317 316 315 314
313 312 311 310 309 308 307 306 305 304 303 302 301 300 299
298 297 296 295 294 293 292 291 290 289 288 287 286 285 284
283 282 281 280 279 278 277 276 275 274 273 272 271 270 269
268 267 266 265 264 263 262 261 260 259 258 257 256 255 254
253 252 251 250 249 248 247 246 245 244 243 242 241 240 239
238 237 236 235 234 233 232 231 230 229 228 227 226 225 224
223 222 221 220 219 218 217 216 215 214 213 212 211 210 209
208 207 206 205 204 203 202 201 200 199 198 197 196 195 194
193 192 191 190 189 188 187 186 185 184 183 182 181 180 179
178 177 176 175 174 173 172 171 170 169 168 167 166 165 164
163 162 161 160 159 158 157 156 155 154 153 152 151 150 149
148 147 146 145 144 143 142 141 140 139 138 137 136 135 134
133 132 131 130 129 128 127 126 125 124 123 122 121 120 119
118 117 116 115 114 113 112 111 110 109 108 107 106 105 104
103 102 101 100 99 98 97 96 95 94 93 92 91 90 89 88 87 86 85
84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65
64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45
44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25
24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2
1 0 succeeded Automatic reboot in 15 seconds - press a key on
the console to abort Rebooting...

nm -n /kernel | grep c01926
c0192600 T m_tag_free
c0192614 T m_tag_prepend
c0192628 T m_tag_unlink
c0192658 T m_tag_delete
c0192674 T m_tag_delete_chain
c01926ac T m_tag_locate
c01926e8 T m_tag_copy

---------------

(kgdb) bt
#0 dumpsys () at ../../kern/kern_shutdown.c:487
#1 0xc0173f3f in boot (howto=256) at ../../kern/kern_shutdown.c:316
#2 0xc0174364 in poweroff_wait (junk=0xc02f2aac,
howto=-1070651985) at ../../kern/kern_shutdown.c:595
#3 0xc02a77ba in trap_fatal (frame=0xd71cbb90,
eva=671510784) at ../../i386/i386/trap.c:974
#4 0xc02a748d in trap_pfault (frame=0xd71cbb90, usermode=0,
eva=671510784) at ../../i386/i386/trap.c:867
#5 0xc02a704b in trap (frame={tf_fs = -1072562160, tf_es =
16, tf_ds = -686030832, tf_edi = 6757530, tf_esi = -1055667456,
tf_ebp = -685982760, tf_isp = -685982788, tf_ebx =
671510784, tf_edx = 6757530, tf_ecx = -1056411648, tf_eax = 28672,
tf_trapno = 12, tf_err = 0, tf_eip = -1072093546, tf_cs
= 8, tf_eflags = 66054, tf_esp = -1055667456, tf_ss = -1055667456})
at ../../i386/i386/trap.c:466
#6 0xc0192696 in m_tag_delete_chain (m=0xc113cb00, t=0x0) at
../../kern/uipc_mbuf2.c:358
#7 0xc01904d3 in m_free (m=0xc113cb00) at
../../kern/uipc_mbuf.c:734
#8 0xc0190606 in m_freem (m=0xc1094000) at
../../kern/uipc_mbuf.c:763
#9 0xc0127df8 in fr_check (ip=0xc1094030, hlen=20,
ifp=0xc21a7008, out=0, mp=0xd71cbce8)
at ../../contrib/ipfilter/netinet/fil.c:1387
#10 0xc01d7d06 in ip_input (m=0xc1094000) at
../../netinet/ip_input.c:478
#11 0xc01d838b in ipintr () at ../../netinet/ip_input.c:971
#12 0xc0299ee9 in swi_net_next ()
#13 0xc016e7c8 in lockmgr (lkp=0xc21a7100, flags=16973826,
interlkp=0xd71c0bac, p=0xd48bd5a0) at ../../kern/kern_lock.c:355
#14 0xc019fda8 in vop_stdlock (ap=0xd71cbdd0) at
../../kern/vfs_default.c:256
#15 0xc025a9c9 in ufs_vnoperatespec (ap=0xd71cbdd0) at
../../ufs/ufs/ufs_vnops.c:2394
#16 0xc01a9ffd in vn_lock (vp=0xd71c0b40, flags=131074,
p=0xd48bd5a0) at vnode_if.h:861
#17 0xc01ad842 in spec_write (ap=0xd71cbe64) at
../../miscfs/specfs/spec_vnops.c:284
#18 0xc025a3ac in ufsspec_write (ap=0xd71cbe64) at
../../ufs/ufs/ufs_vnops.c:1827
#19 0xc025a9c9 in ufs_vnoperatespec (ap=0xd71cbe64) at
../../ufs/ufs/ufs_vnops.c:2394 #20 0xc01a9b9a in vn_write
(fp=0xc219b100, uio=0xd71cbed4, cred=0xc2197080, flags=0,
p=0xd48bd5a0) at vnode_if.h:363
#21 0xc018330d in dofilewrite (p=0xd48bd5a0, fp=0xc219b100,
fd=9, buf=0xbfbfe89c, nbyte=580, offset=-1, flags=0)
at ../../sys/file.h:163
#22 0xc01831c6 in write (p=0xd48bd5a0, uap=0xd71cbf80) at
../../kern/sys_generic.c:329
#23 0xc02a7a69 in syscall2 (frame={tf_fs = -1078001617, tf_es
= 134938671, tf_ds = -1078001617, tf_edi = 135090176, tf_esi = 580,
tf_ebp = -1077940064, tf_isp = -685981740, tf_ebx =
-1077942112, tf_edx = 0, tf_ecx = 13, tf_eax = 4, tf_trapno = 7,
tf_err = 2, tf_eip = 673683504, tf_cs = 31, tf_eflags =
663, tf_esp = -1077942172, tf_ss = 47})
at ../../i386/i386/trap.c:1175
#24 0xc0298a85 in Xint0x80_syscall ()
#25 0x80655de in ?? ()
#26 0x806c2fb in ?? ()
#27 0x806c21d in ?? ()
#28 0x807470a in ?? ()
#29 0x8083a78 in ?? ()
#30 0x805b84b in ?? ()
#31 0x804d484 in ?? ()
#32 0x806ed77 in ?? ()
#33 0x806e967 in ?? ()
#34 0x804b62a in ?? ()

(kgdb) f 11
#11 0xc01d838b in ipintr () at ../../netinet/ip_input.c:971
971 ip_input(m);
(kgdb) print *m
$13 = {m_hdr = {mh_next = 0xc1128100, mh_nextpkt = 0x0,
mh_data = 0xc1094030 "E", mh_len = 208, mh_type = 0, mh_flags = 2},
M_dat = {MH = {MH_pkthdr = {rcvif = 0xc21a7008, len = 576,
header = 0x0, csum_flags = 0, csum_data = 0, tags = {
slh_first = 0x0}}, MH_dat = {MH_ext = {ext_buf =
0x2000000 <Address 0x2000000 out of bounds>, ext_free = 0x2400045,
ext_size = 28454, ext_ref = 0x6d3d012e},
MH_databuf =
"\000\000\000\002E\000@\002&o\000\000.\001=m;½ó0ض\037=\003\00
1º\006\000\000\000\000E\000\005Èéº@\000/\006¢tض\037=À¨\001e'B
\rmy»6\216^\222@\002P\020á\000±¡\000\000\000\000@\t\a\000\000\
002r\000\003À\000\227«\212!\225@\204]\001\214\027Áù\177\232ùãò
\222\016\000ùau1=\216Ý\204¨\207O\002+ø0éHð\eD\2056n\001÷U\025à
\222þ\f:SõPI\037)TØ(ý¨\rÓ@\210ê\217(Sõcâé¸\n?\217%x&µ\177ôUqX\
222\020\225\\ÑÙ~\fhê©\036\t\"Az\206ápþ+}Ç££¢"...}},
M_databuf = "\bp\032Â@\002", '\000' <repeats 21 times>,
"\002E\000@\002&o\000\000.\001=m;½ó0ض\037=\003\001º\006\000\0
00\000\000E\000\005Èéº@\000/\006¢tض\037=À¨\001e'B\rmy»6\216^\
222@\002P\020á\000±¡\000\000\000\000@\t\a\000\000\002r\000\003
À\000\227«\212!\225@\204]\001\214\027Áù\177\232ùãò\222\016\000
ùau1=\216Ý\204¨\207O\002+ø0éHð\eD\2056n\001÷U\025à\222þ\f:SõPI
\037)TØ(ý¨\rÓ@\210ê\217(Sõcâé¸\n?\217%x&µ\177ôUqX\222\020\225\
\ÑÙ~\fhê©\036\t"...}}
(kgdb) f 6
#6 0xc0192696 in m_tag_delete_chain (m=0xc113cb00, t=0x0) at
../../kern/uipc_mbuf2.c:358
358 m_tag_delete(m, q);
(kgdb) print *m
$14 = {m_hdr = {mh_next = 0xc113ca00, mh_nextpkt =
0x280ef4cd, mh_data = 0x14 <Address 0x14 out of bounds>,
mh_len = 663,
mh_type = 28672, mh_flags = 10246}, M_dat = {MH =
{MH_pkthdr = {rcvif = 0x280ef492, len = 672120748, header = 0x2,
csum_flags = 16384, csum_data = 1, tags = {slh_first
= 0x28067100}}, MH_dat = {MH_ext = {
ext_buf = 0x28067200 <Address 0x28067200 out of
bounds>, ext_free = 0x280541fd, ext_size = 134516476,
ext_ref = 0x280819da},
MH_databuf =
"\000r\006(ýA\005(ü\216\004\bÚ\031\b(\000\000\000\000¢A\005(¨:
\006(@\200\006(\000\000\000\000\000\000\000\000`û¿¿@\200\006\0
01\234û¿¿OA\005(ü\216\004\b\004Ïe\000\000r\006(\001\000\000\00
0¨:\006(\000p\006(ü\216\004\býA\005(ü\216\004\b\t\013\005(\200
æ\020(¢A\005(¨:\006(\000é\a(\200=\006(\227?\005(5(\005(¨:\006(
\fü¿¿Ï@\005(
\006\005(\004Ïe\000\200=\006(\001\000\000\000\000p\006(\000q\0
06(\000r\006(Ú>\005(¨:\006(\000p\006(ü\216\004\bÐü¿¿SÔ\004\b\2
00æ\020(@ \005\b\000r\006("...}},
M_databuf =
"\222ô\016(¬¿\017(\002\000\000\000\000@\000\000\001\000\000\00
0\000q\006(\000r\006(ýA\005(ü\216\004\bÚ\031\b(\000\000\000\00
0¢A\005(¨:\006(@\200\006(\000\000\000\000\000\000\000\000`û¿¿@
\200\006\001\234û¿¿OA\005(ü\216\004\b\004Ïe\000\000r\006(\001\
000\000\000¨:\006(\000p\006(ü\216\004\býA\005(ü\216\004\b\t\01
3\005(\200æ\020(¢A\005(¨:\006(\000é\a(\200=\006(\227?\005(5(\0
05(¨:\006(\fü¿¿Ï@\005(
\006\005(\004Ïe\000\200=\006(\001\000\000\000\000p\006(\000q\0
06(\000r\006(Ú>\005(¨:\006(\000p\006("...}}
(kgdb) print *q
Cannot access memory at address 0x0.
(kgdb)
----------------------------
Copyright (c) 1992-2005 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992,
1993, 1994
The Regents of the University of California. All
rights reserved.
FreeBSD 4.11-RELEASE-p13 #3: Thu Feb 23 13:09:31 EST 2006
damascus@xxxxxxxxxxxxxxxxxxxxx:/usr/src/sys/compile/DAEMON
Timecounter "i8254" frequency 1193182 Hz
CPU: Intel(R) Pentium(R) 4 CPU 2.00GHz (1993.54-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf24 Stepping = 4

Features=0x3febfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SE
P,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SS
E2,SS,HTT,TM>
real memory = 536608768 (524032K bytes) avail memory =
518377472 (506228K bytes) Preloaded elf kernel "kernel" at
0xc03af000.
Warning: Pentium 4 CPU: PSE disabled
Pentium Pro MTRR support enabled
md0: Malloc disk
Using $PIR table, 11 entries at 0xc00f28c0
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Intel 82845 Host to PCI bridge> on motherboard
pci0: <PCI bus> on pcib0
pcib1: <Intel 82845 PCI-PCI (AGP) bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
pcib2: <Intel 82801BA/BAM (ICH2) Hub to PCI bridge> at device
30.0 on pci0
pci2: <PCI bus> on pcib2
twe0: <3ware Storage Controller driver ver. 1.40.01.002> port
0xdfa0-0xdfaf mem 0xfe000000-0xfe7fffff,0xfeafec00-0xfeafec0f
irq 9 at device 9.0 on pci2
twe0: 4 ports, Firmware FE7X 1.05.00.068, BIOS BE7X 1.08.00.048
fxp0: <Intel 82550 Pro/100 Ethernet> port 0xdf00-0xdf3f mem
0xfeaa0000-0xfeabffff,0xfeafd000-0xfeafdfff irq 11 at device
10.0 on pci2
fxp0: Ethernet address 00:02:b3:d0:e3:73
inphy0: <i82555 10/100 media interface> on miibus0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: <Intel 82550 Pro/100 Ethernet> port 0xde80-0xdebf mem
0xfea80000-0xfea9ffff,0xfeafc000-0xfeafcfff irq 10 at device
11.0 on pci2
fxp1: Ethernet address 00:02:b3:ee:65:88
inphy1: <i82555 10/100 media interface> on miibus1
inphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp2: <Intel 82550 Pro/100 Ethernet> port 0xdd80-0xddbf mem
0xfea40000-0xfea5ffff,0xfeafb000-0xfeafbfff irq 11 at device
12.0 on pci2
fxp2: Ethernet address 00:11:11:c1:a2:e5
inphy2: <i82555 10/100 media interface> on miibus2
inphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp3: <Intel 82550 Pro/100 Ethernet> port 0xdd00-0xdd3f mem
0xfea20000-0xfea3ffff,0xfeafa000-0xfeafafff irq 11 at device
13.0 on pci2
fxp3: Ethernet address 00:11:11:c1:a2:e7
inphy3: <i82555 10/100 media interface> on miibus3
inphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
atapci0: <Promise ATA100 controller> port
0xdcc0-0xdcff,0xdfe0-0xdfe3,0xdf98-0xdf9f,0xdfe4-0xdfe7,0xdff0
-0xdff7 mem 0xfe9e0000-0xfe9fffff irq 11 at device 14.0 on pci2
ata2: at 0xdff0 on atapci0
ata3: at 0xdf98 on atapci0
pci2: <ATI Mach64-GR graphics accelerator> at 15.0 irq 11
isab0: <Intel 82801BA/BAM (ICH2) PCI to LPC bridge> at device
31.0 on pci0
isa0: <ISA bus> on isab0
atapci1: <Intel ICH2 ATA100 controller> port 0xffa0-0xffaf at
device 31.1 on pci0
ata0: at 0x1f0 irq 14 on atapci1
ata1: at 0x170 irq 15 on atapci1
uhci0: <Intel 82801BA/BAM (ICH2) USB controller USB-A> port
0xef40-0xef5f irq 11 at device 31.2 on pci0
usb0: <Intel 82801BA/BAM (ICH2) USB controller USB-A> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
pci0: <unknown card> (vendor=0x8086, dev=0x2443) at 31.3 irq 11
uhci1: <Intel 82801BA/BAM (ICH2) USB controller USB-B> port
0xef80-0xef9f irq 10 at device 31.4 on pci0
usb1: <Intel 82801BA/BAM (ICH2) USB controller USB-B> on uhci1
usb1: USB revision 1.0
uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
orm0: <Option ROMs> at iomem
0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9800-0xcafff,0xcb000-0xcbff
f,0xcc000-0xccfff on isa0 pmtimer0 on isa0
fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6
drq 2 on isa0
fdc0: FIFO enabled, 8 bytes threshold
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: model VersaPad, device ID 0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem
0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100> sio0 at port
0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A, console
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode
ppbus0: IEEE1284 device found /NIBBLE/PS2/ECP Probing for PnP
devices on ppbus0:
ppbus0: <Lexmark International Lexmark E323> PRINTER PCL 6
Emulation, PostScript Level 3 Emulation, NPAP, PJL
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
DUMMYNET initialized (011031)
IP packet filtering initialized, divert enabled, rule-based
forwarding enabled, default to deny, unlimited logging IP
Filter: v3.4.35 initialized. Default = pass all, Logging = enabled
acd0: CDROM <GCR-8521B> at ata0-master PIO4
twed0: <Unit 0, TwinStor, Normal> on twe0
twed0: 152626MB (312579760 sectors)
twed1: <Unit 2, JBOD, Normal> on twe0
twed1: 78167MB (160086528 sectors)
twed2: <Unit 3, JBOD, Normal> on twe0
twed2: 152627MB (312581808 sectors)
Mounting root from ufs:/dev/twed0s1a
WARNING: / was not properly dismounted
-------------------------------------------------
daemon# cat /usr/src/sys/i386/conf/DAEMON # # GENERIC --
Generic kernel configuration file for FreeBSD/i386 # # For
more information on this file, please read the handbook
section on # Kernel Configuration Files:
#
#
http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kern
elconfig-config.html
#
# The handbook is also available locally in
/usr/share/doc/handbook # if you've installed the doc
distribution, otherwise always see the # FreeBSD World Wide
Web server (http://www.FreeBSD.org/) for the # latest information.
#
# An exhaustive list of options and more detailed
explanations of the # device lines is also present in the
./LINT configuration file. If you are # in doubt as to the
purpose or necessity of a line, check first in LINT.
#
# $FreeBSD: src/sys/i386/conf/GENERIC,v 1.246.2.48 2002/08/31
20:28:26 obrien Exp $

machine i386
cpu I386_CPU
cpu I486_CPU
cpu I586_CPU
cpu I686_CPU
ident "DAEMON"
maxusers 0

makeoptions DEBUG=-g #Build kernel with
gdb(1) debug symbols

#options DDB
options MATH_EMULATE #Support for x87 emulation
options INET #InterNETworking
options INET6 #IPv6
communications protocols
options FFS #Berkeley Fast Filesystem
options FFS_ROOT #FFS usable as root
device [keep this!]
options SOFTUPDATES #Enable FFS soft
updates support
options UFS_DIRHASH #Improve performance
on big directories
options MFS #Memory Filesystem
options MD_ROOT #MD is a potential
root device
options NFS #Network Filesystem
options NFS_ROOT #NFS usable as root
device, NFS required
options MSDOSFS #MSDOS Filesystem
options CD9660 #ISO 9660 Filesystem
options CD9660_ROOT #CD-ROM usable as
root, CD9660 required
options PROCFS #Process filesystem
options COMPAT_43 #Compatible with BSD
4.3 [KEEP THIS!]
options SCSI_DELAY=15000 #Delay (in ms) before
probing SCSI
options UCONSOLE #Allow users to grab
the console
options USERCONFIG #boot -c editor
options VISUAL_USERCONFIG #visual boot -c editor
options KTRACE #ktrace(1) support
options SYSVSHM #SYSV-style shared memory
options SYSVMSG #SYSV-style message queues
options SYSVSEM #SYSV-style semaphores
options P1003_1B #Posix P1003_1B
real-time extensions
options _KPOSIX_PRIORITY_SCHEDULING
options ICMP_BANDLIM #Rate limit bad replies
options KBD_INSTALL_CDEV # install a CDEV
entry in /dev
options AHC_REG_PRETTY_PRINT # Print register
bitfields in debug
# output. Adds ~128k
to driver.
options AHD_REG_PRETTY_PRINT # Print register
bitfields in debug
# output. Adds ~215k
to driver.

# To make an SMP kernel, the next two are needed
#options SMP # Symmetric
MultiProcessor Kernel
#options APIC_IO # Symmetric (APIC) I/O

device isa
device pci

# Floppy drives
device fdc0 at isa? port IO_FD1 irq 6 drq 2
device fd0 at fdc0 drive 0
device fd1 at fdc0 drive 1
#
# If you have a Toshiba Libretto with its Y-E Data PCMCIA
floppy, # don't use the above line for fdc0 but the following one:
#device fdc0

# ATA and ATAPI devices
device ata0 at isa? port IO_WD1 irq 14
device ata1 at isa? port IO_WD2 irq 15
device ata
device atadisk # ATA disk drives
device atapicd # ATAPI CDROM drives
device atapifd # ATAPI floppy drives
device atapist # ATAPI tape drives
options ATA_STATIC_ID #Static device numbering

# RAID controllers
device twe # 3ware Escalade

# atkbdc0 controls both the keyboard and the PS/2 mouse
device atkbdc0 at isa? port IO_KBD
device atkbd0 at atkbdc? irq 1
device psm0 at atkbdc? irq 12

device vga0 at isa?

# splash screen/screen saver
pseudo-device splash

# syscons is the default console driver, resembling an SCO console
device sc0 at isa? flags 0x100

# Floating point support - do not disable.
device npx0 at nexus? port IO_NPX irq 13

# Power management support (see LINT for more options)
device apm0 at nexus? disable flags 0x20 #
Advanced Power Management

# Serial (COM) ports
device sio0 at isa? port IO_COM1 flags 0x10 irq 4
device sio1 at isa? port IO_COM2 irq 3

# Parallel port
device ppc0 at isa? irq 7
device ppbus # Parallel port bus (required)
device lpt # Printer
device plip # TCP/IP over parallel
device ppi # Parallel port interface device
#device vpo # Requires scbus and da


# PCI Ethernet NICs.
device de # DEC/Intel DC21x4x (``Tulip'')

# PCI Ethernet NICs that use the common MII bus controller code.
# NOTE: Be sure to keep the 'device miibus' line in order to
use these NICs!
device miibus # MII bus support
device fxp # Intel EtherExpress PRO/100B
(82557, 82558)

# Pseudo devices - the number indicates how many units to allocate.
pseudo-device loop # Network loopback
pseudo-device ether # Ethernet support
pseudo-device sl 1 # Kernel SLIP
pseudo-device ppp 1 # Kernel PPP
pseudo-device tun # Packet tunnel.
pseudo-device pty # Pseudo-ttys (telnet etc)
pseudo-device md # Memory "disks"
pseudo-device gif # IPv6 and IPv4 tunneling
pseudo-device faith 1 # IPv6-to-IPv4 relaying
(translation)

# The `bpf' pseudo-device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
pseudo-device bpf #Berkeley packet filter

# USB support
device uhci # UHCI PCI->USB interface
device ohci # OHCI PCI->USB interface
device usb # USB Bus (required)
device ugen # Generic
device uhid # "Human Interface Devices"
device ukbd # Keyboard
device ulpt # Printer
device ums # Mouse
device uscanner # Scanners
device urio # Diamond Rio MP3 Player
# USB Ethernet, requires mii
device aue # ADMtek USB ethernet
device cue # CATC USB ethernet
device kue # Kawasaki LSI USB ethernet

#NATD
options IPFIREWALL
options IPDIVERT
options IPFIREWALL_FORWARD

#DUMMYNET
options IPFIREWALL_VERBOSE
options DUMMYNET

#NMBCLUSTER
options NMBCLUSTERS=5120

#IPFILTER
options IPFILTER
options IPFILTER_LOG

#SysV Memory

options SEMMAP=31
options SEMMNI=70
options SEMMNS=200
options SEMMNU=31
options SEMMSL=61
options SEMOPM=101
options SEMUME=11
options SHMMAXPGS=16384
options SHMSEG=10

#pseudo
pseudo-device vn
pseudo-device snp 3

options CONSPEED=115200 #default speed for
serial console (default 9600)
#options IPSEC #IP security
#options IPSEC_ESP #IP security
(crypto; define w/ IPSEC)
#options IPSEC_DEBUG #debug for IP security

options NETGRAPH #netgraph(4) system
options NETGRAPH_PPPOE
options NETGRAPH_SOCKET
options NETGRAPH_ETHER
-----------------
FreeBSD daemon.faerunhome.com 4.11-RELEASE-p13 FreeBSD
4.11-RELEASE-p13 #3: Thu Feb 23 13:09:31 EST 2006
damascus@xxxxxxxxxxxxxxxxxxxxx:/usr/src/sys/compile/DAEMON i386





- Carroll Kong


_______________________________________________
freebsd-hackers@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to
"freebsd-hackers-unsubscribe@xxxxxxxxxxx"


_______________________________________________
freebsd-hackers@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • RE: FreeBSD 4.11 P13 Crash
    ... It happened again even with a new CPU and new PowerSupply. ... IPFilter, ... page fault while in kernel mode ... Okay this time my kernel was recompiled so there are no ...
    (freebsd-hackers)
  • Fw: FreeBSD 6.2 Repeating Crash - Sleeping thread; Fatal trap 12: page fault; warning: T2 might
    ... This crash is occurring on a dual-AMD 1.6Ghz cpu white-box system with 1 Gb ram, 250Gb storage running GENERIC kernel. ... About 3 - 4 months ago, the system was upgraded from an earlier FreeBSD version to 6.1. ... page fault while in kernel mode ... We attempted to use DDB to analyze the dumps (struggling now, unfamiliar with kernel debugging process) with no success. ...
    (freebsd-questions)
  • kernel panic when using safe(4)
    ... I am trying to install SafeNet 1141 support in one of the freebsd ... kernel config and compile to enable hardware crypto acceleration. ... page fault while in kernel mode ... FreeBSD is a registered trademark of The FreeBSD Foundation. ...
    (freebsd-net)
  • FreeBSD 4.11 P13 Crash
    ... Okay this time my kernel was recompiled so there are no modules to make it ... logging with IPFilter, but I am always open to the possibility. ... page fault while in kernel mode ... pseudo-device sl 1 # Kernel SLIP ...
    (freebsd-hackers)
  • Re: IpFilter / IpFireWall
    ... to use ipfw&ipf use this in your kernel! ... Subject: IpFilter / IpFireWall ... > i use freebsd v4.5 Release #0. ...
    (FreeBSD-Security)