Fw: LOR:vfs_bio.c and ufs_dirhash.c




--- On Mon, 15/6/09, Ivailo Bonev <ibb27@xxxxxxxxxxx> wrote:

From: Ivailo Bonev <ibb27@xxxxxxxxxxx>
Subject: LOR:vfs_bio.c and ufs_dirhash.c
To: freebsd-current@xxxxxxxxxxx
Date: Monday, 15 June, 2009, 7:20 PM
I've installed 8-CURRENT on my HP
laptop, snapshot from pub.allbsd.org with HEAD sources from
yesterday, but on first startup I see LOR. Everything works
ok, though...
Here is log from messages:
kernel: Copyright (c) 1992-2009 The FreeBSD Project.
kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989,
1991, 1992, 1993, 1994
kernel: The Regents of the University of California. All
rights reserved.
kernel: FreeBSD is a registered trademark of The FreeBSD
Foundation.
kernel: FreeBSD 8.0-HEAD-20090615-JPSNAP #0: Mon Jun 15
02:22:17 UTC 2009
kernel: root@xxxxxxxxxxxxxxxxxxxxxxxxxxxx:/usr/obj/i386/usr/src/sys/GENERIC
kernel: WARNING: WITNESS option enabled, expect reduced
performance.
kernel: Timecounter "i8254" frequency 1193182 Hz quality 0
Jun 15 17:42:22 ibb kernel: CPU: Pentium(R) Dual-Core
CPU       T4200  @ 2.00GHz
(1995.25-MHz 686-class CPU)
Jun 15 17:42:22 ibb kernel: Origin = "GenuineIntel" 
Id = 0x1067a  Stepping = 10
Jun 15 17:42:22 ibb kernel:
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Jun 15 17:42:22 ibb kernel:
Features2=0x400e39d<SSE3,DTES64,MON,DS_CPL,EST,TM2,SSSE3,CX16,xTPR,PDCM,XSAVE>
Jun 15 17:42:22 ibb kernel: AMD
Features=0x20100000<NX,LM>
Jun 15 17:42:22 ibb kernel: AMD Features2=0x1<LAHF>
Jun 15 17:42:22 ibb kernel: TSC: P-state invariant
Jun 15 17:42:22 ibb kernel: real memory  = 2147483648
(2048 MB)
Jun 15 17:42:22 ibb kernel: avail memory = 2083385344 (1986
MB)
Jun 15 17:42:22 ibb kernel: ACPI APIC Table: <HPQOEM
30E9    >
Jun 15 17:42:22 ibb kernel: FreeBSD/SMP: Multiprocessor
System Detected: 2 CPUs
Jun 15 17:42:22 ibb kernel: FreeBSD/SMP: 1 package(s) x 2
core(s)
Jun 15 17:42:22 ibb kernel: cpu0 (BSP): APIC ID:  0
Jun 15 17:42:22 ibb kernel: cpu1 (AP): APIC ID:  1
Jun 15 17:42:22 ibb kernel: ioapic0: Changing APIC ID to 1
Jun 15 17:42:22 ibb kernel: ioapic0 <Version 2.0>
irqs 0-23 on motherboard
Jun 15 17:42:22 ibb kernel: kbd1 at kbdmux0
Jun 15 17:42:22 ibb kernel: acpi0: <HPQOEM SLIC-MPC>
on motherboard
Jun 15 17:42:22 ibb kernel: acpi0: [ITHREAD]
Jun 15 17:42:22 ibb kernel: acpi0: Power Button (fixed)
Jun 15 17:42:22 ibb kernel: Timecounter "HPET" frequency
14318180 Hz quality 900
Jun 15 17:42:22 ibb kernel: Timecounter "ACPI-fast"
frequency 3579545 Hz quality 1000
Jun 15 17:42:22 ibb kernel: acpi_timer0: <24-bit timer
at 3.579545MHz> port 0x408-0x40b on acpi0
Jun 15 17:42:22 ibb kernel: acpi_ec0: <Embedded
Controller: GPE 0x16> port 0x62,0x66 on acpi0
Jun 15 17:42:22 ibb kernel: pcib0: <ACPI Host-PCI
bridge> port 0xcf8-0xcff on acpi0
Jun 15 17:42:22 ibb kernel: pci0: <ACPI PCI bus> on
pcib0
Jun 15 17:42:22 ibb kernel: pcib1: <ACPI PCI-PCI
bridge> irq 16 at device 1.0 on pci0
Jun 15 17:42:22 ibb kernel: pci1: <ACPI PCI bus> on
pcib1
Jun 15 17:42:22 ibb kernel: vgapci0: <VGA-compatible
display> port 0x7000-0x70ff mem
0x80000000-0x8fffffff,0x98400000-0x9840ffff irq 16 at device
0.0 on pci1
Jun 15 17:42:22 ibb kernel: uhci0: <Intel 82801I (ICH9)
USB controller> port 0x80c0-0x80df irq 16 at device 26.0
on pci0
Jun 15 17:42:22 ibb kernel: uhci0: [ITHREAD]
Jun 15 17:42:22 ibb kernel: uhci0: LegSup = 0x0f00
Jun 15 17:42:22 ibb kernel: usbus0: <Intel 82801I (ICH9)
USB controller> on uhci0
Jun 15 17:42:22 ibb kernel: uhci1: <Intel 82801I (ICH9)
USB controller> port 0x80a0-0x80bf irq 17 at device 26.1
on pci0
Jun 15 17:42:22 ibb kernel: uhci1: [ITHREAD]
Jun 15 17:42:22 ibb kernel: uhci1: LegSup = 0x0f00
Jun 15 17:42:22 ibb kernel: usbus1: <Intel 82801I (ICH9)
USB controller> on uhci1
Jun 15 17:42:22 ibb kernel: uhci2: <Intel 82801I (ICH9)
USB controller> port 0x8080-0x809f irq 18 at device 26.2
on pci0
Jun 15 17:42:22 ibb kernel: uhci2: [ITHREAD]
Jun 15 17:42:22 ibb kernel: uhci2: LegSup = 0x0f00
Jun 15 17:42:22 ibb kernel: usbus2: <Intel 82801I (ICH9)
USB controller> on uhci2
Jun 15 17:42:22 ibb kernel: ehci0: <Intel 82801I (ICH9)
USB 2.0 controller> mem 0x98504c00-0x98504fff irq 19 at
device 26.7 on pci0
Jun 15 17:42:22 ibb kernel: ehci0: [ITHREAD]
Jun 15 17:42:22 ibb kernel: usbus3: EHCI version 1.0
Jun 15 17:42:22 ibb kernel: usbus3: <Intel 82801I (ICH9)
USB 2.0 controller> on ehci0
Jun 15 17:42:22 ibb kernel: pci0: <multimedia, HDA>
at device 27.0 (no driver attached)
Jun 15 17:42:22 ibb kernel: pcib2: <ACPI PCI-PCI
bridge> irq 16 at device 28.0 on pci0
Jun 15 17:42:22 ibb kernel: pci2: <ACPI PCI bus> on
pcib2
Jun 15 17:42:22 ibb kernel: pcib3: <ACPI PCI-PCI
bridge> irq 17 at device 28.1 on pci0
Jun 15 17:42:22 ibb kernel: pci3: <ACPI PCI bus> on
pcib3
Jun 15 17:42:22 ibb kernel: pci3: <network> at device
0.0 (no driver attached)
Jun 15 17:42:22 ibb kernel: pcib4: <ACPI PCI-PCI
bridge> irq 18 at device 28.2 on pci0
Jun 15 17:42:22 ibb kernel: pci4: <ACPI PCI bus> on
pcib4
Jun 15 17:42:22 ibb kernel: pcib5: <ACPI PCI-PCI
bridge> irq 16 at device 28.4 on pci0
Jun 15 17:42:22 ibb kernel: pci69: <ACPI PCI bus> on
pcib5
Jun 15 17:42:22 ibb kernel: pcib6: <ACPI PCI-PCI
bridge> irq 17 at device 28.5 on pci0
Jun 15 17:42:22 ibb kernel: pci134: <ACPI PCI bus> on
pcib6
Jun 15 17:42:22 ibb kernel: mskc0: <Marvell Yukon
88E8072 Gigabit Ethernet> port 0x2000-0x20ff mem
0x90100000-0x90103fff irq 17 at device 0.0 on pci134
Jun 15 17:42:22 ibb kernel: msk0: <Marvell Technology
Group Ltd. Yukon EX Id 0xb5 Rev 0x02> on mskc0
Jun 15 17:42:22 ibb kernel: msk0: Ethernet address:
00:24:81:47:09:c0
Jun 15 17:42:22 ibb kernel: miibus0: <MII bus> on
msk0
Jun 15 17:42:22 ibb kernel: e1000phy0: <Marvell 88E1149
Gigabit PHY> PHY 0 on miibus0
Jun 15 17:42:22 ibb kernel: e1000phy0:  10baseT,
10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto
Jun 15 17:42:22 ibb kernel: mskc0: [FILTER]
Jun 15 17:42:22 ibb kernel: uhci3: <Intel 82801I (ICH9)
USB controller> port 0x8060-0x807f irq 20 at device 29.0
on pci0
Jun 15 17:42:22 ibb kernel: uhci3: [ITHREAD]
Jun 15 17:42:22 ibb kernel: uhci3: LegSup = 0x0f00
Jun 15 17:42:22 ibb kernel: usbus4: <Intel 82801I (ICH9)
USB controller> on uhci3
Jun 15 17:42:22 ibb kernel: uhci4: <Intel 82801I (ICH9)
USB controller> port 0x8040-0x805f irq 22 at device 29.1
on pci0
Jun 15 17:42:22 ibb kernel: uhci4: [ITHREAD]
Jun 15 17:42:22 ibb kernel: uhci4: LegSup = 0x0f00
Jun 15 17:42:22 ibb kernel: usbus5: <Intel 82801I (ICH9)
USB controller> on uhci4
Jun 15 17:42:22 ibb kernel: uhci5: <Intel 82801I (ICH9)
USB controller> port 0x8020-0x803f irq 18 at device 29.2
on pci0
Jun 15 17:42:22 ibb kernel: uhci5: [ITHREAD]
Jun 15 17:42:22 ibb kernel: uhci5: LegSup = 0x0f00
Jun 15 17:42:22 ibb kernel: usbus6: <Intel 82801I (ICH9)
USB controller> on uhci5
Jun 15 17:42:22 ibb kernel: ehci1: <Intel 82801I (ICH9)
USB 2.0 controller> mem 0x98504800-0x98504bff irq 20 at
device 29.7 on pci0
Jun 15 17:42:22 ibb kernel: ehci1: [ITHREAD]
Jun 15 17:42:22 ibb kernel: usbus7: EHCI version 1.0
Jun 15 17:42:22 ibb kernel: usbus7: <Intel 82801I (ICH9)
USB 2.0 controller> on ehci1
Jun 15 17:42:22 ibb kernel: pcib7: <ACPI PCI-PCI
bridge> at device 30.0 on pci0
Jun 15 17:42:22 ibb kernel: pci135: <ACPI PCI bus> on
pcib7
Jun 15 17:42:22 ibb kernel: isab0: <PCI-ISA bridge>
at device 31.0 on pci0
Jun 15 17:42:22 ibb kernel: isa0: <ISA bus> on isab0
Jun 15 17:42:22 ibb kernel: atapci0: <Intel AHCI
controller> port
0x80e8-0x80ef,0x80f4-0x80f7,0x80e0-0x80e7,0x80f0-0x80f3,0x8000-0x801f
mem 0x98504000-0x985047ff irq 21 at device 31.2 on pci0
Jun 15 17:42:22 ibb kernel: atapci0: [ITHREAD]
Jun 15 17:42:22 ibb kernel: atapci0: AHCI v1.20 controller
with 4 3Gbps ports, PM not supported
Jun 15 17:42:22 ibb kernel: ata2: <ATA channel 0> on
atapci0
Jun 15 17:42:22 ibb kernel: ata2: [ITHREAD]
Jun 15 17:42:22 ibb kernel: ata3: <ATA channel 1> on
atapci0
Jun 15 17:42:22 ibb kernel: ata3: [ITHREAD]
Jun 15 17:42:22 ibb kernel: ata4: <ATA channel 5> on
atapci0
Jun 15 17:42:22 ibb kernel: ata4: [ITHREAD]
Jun 15 17:42:22 ibb kernel: battery0: <ACPI Control
Method Battery> on acpi0
Jun 15 17:42:22 ibb kernel: acpi_acad0: <AC Adapter>
on acpi0
Jun 15 17:42:22 ibb kernel: acpi_button0: <Sleep
Button> on acpi0
Jun 15 17:42:22 ibb kernel: acpi_lid0: <Control Method
Lid Switch> on acpi0
Jun 15 17:42:22 ibb kernel: acpi_tz0: <Thermal Zone>
on acpi0
Jun 15 17:42:22 ibb kernel: acpi_tz1: <Thermal Zone>
on acpi0
Jun 15 17:42:22 ibb kernel: acpi_tz1: _CRT value is absurd,
ignored (256.0C)
Jun 15 17:42:22 ibb kernel: acpi_tz2: <Thermal Zone>
on acpi0
Jun 15 17:42:22 ibb kernel: acpi_tz3: <Thermal Zone>
on acpi0
Jun 15 17:42:22 ibb kernel: acpi_tz4: <Thermal Zone>
on acpi0
Jun 15 17:42:22 ibb kernel: acpi_tz5: <Thermal Zone>
on acpi0
Jun 15 17:42:22 ibb kernel: atrtc0: <AT realtime
clock> port 0x70-0x77 irq 8 on acpi0
Jun 15 17:42:22 ibb kernel: atrtc0: Warning: Couldn't map
I/O.
Jun 15 17:42:22 ibb kernel: atkbdc0: <Keyboard
controller (i8042)> port 0x60,0x64 irq 1 on acpi0
Jun 15 17:42:22 ibb kernel: atkbd0: <AT Keyboard> irq
1 on atkbdc0
Jun 15 17:42:22 ibb kernel: kbd0 at atkbd0
Jun 15 17:42:22 ibb kernel: atkbd0: [GIANT-LOCKED]
Jun 15 17:42:22 ibb kernel: atkbd0: [ITHREAD]
Jun 15 17:42:22 ibb kernel: psm0: <PS/2 Mouse> irq 12
on atkbdc0
Jun 15 17:42:22 ibb kernel: psm0: [GIANT-LOCKED]
Jun 15 17:42:22 ibb kernel: psm0: [ITHREAD]
Jun 15 17:42:22 ibb kernel: psm0: model IntelliMouse,
device ID 3
Jun 15 17:42:22 ibb kernel: cpu0: <ACPI CPU> on
acpi0
Jun 15 17:42:22 ibb kernel: est0: <Enhanced SpeedStep
Frequency Control> on cpu0
Jun 15 17:42:22 ibb kernel: p4tcc0: <CPU Frequency
Thermal Control> on cpu0
Jun 15 17:42:22 ibb kernel: cpu1: <ACPI CPU> on
acpi0
Jun 15 17:42:22 ibb kernel: est1: <Enhanced SpeedStep
Frequency Control> on cpu1
Jun 15 17:42:22 ibb kernel: p4tcc1: <CPU Frequency
Thermal Control> on cpu1
Jun 15 17:42:22 ibb kernel: pmtimer0 on isa0
Jun 15 17:42:22 ibb kernel: sc0: <System console> at
flags 0x100 on isa0
Jun 15 17:42:22 ibb kernel: sc0: VGA <16 virtual
consoles, flags=0x300>
Jun 15 17:42:22 ibb kernel: vga0: <Generic ISA VGA>
at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Jun 15 17:42:22 ibb kernel: ata0 at port 0x1f0-0x1f7,0x3f6
irq 14 on isa0
Jun 15 17:42:22 ibb kernel: ata0: [ITHREAD]
Jun 15 17:42:22 ibb kernel: ata1 at port 0x170-0x177,0x376
irq 15 on isa0
Jun 15 17:42:22 ibb kernel: ata1: [ITHREAD]
Jun 15 17:42:22 ibb kernel: Timecounters tick every 1.000
msec
Jun 15 17:42:22 ibb kernel: usbus1: 12Mbps Full Speed USB
v1.0
Jun 15 17:42:22 ibb kernel: usbus2: 12Mbps Full Speed USB
v1.0
Jun 15 17:42:22 ibb kernel: usbus3: 480Mbps High Speed USB
v2.0
Jun 15 17:42:22 ibb kernel: usbus4: 12Mbps Full Speed USB
v1.0
Jun 15 17:42:22 ibb kernel: usbus5: 12Mbps Full Speed USB
v1.0
Jun 15 17:42:22 ibb kernel: usbus6: 12Mbps Full Speed USB
v1.0
Jun 15 17:42:22 ibb kernel: usbus7: 480Mbps High Speed USB
v2.0
Jun 15 17:42:22 ibb kernel: usbus0: 12Mbps Full Speed USB
v1.0
Jun 15 17:42:22 ibb kernel: ugen1.1: <Intel> at
usbus1
Jun 15 17:42:22 ibb kernel: uhub0: <Intel UHCI root HUB,
class 9/0, rev 1.00/1.00, addr 1> on usbus1
Jun 15 17:42:22 ibb kernel: ugen2.1: <Intel> at
usbus2
Jun 15 17:42:22 ibb kernel: uhub1: <Intel UHCI root HUB,
class 9/0, rev 1.00/1.00, addr 1> on usbus2
Jun 15 17:42:22 ibb kernel: ugen3.1: <Intel> at
usbus3
Jun 15 17:42:22 ibb kernel: uhub2: <Intel EHCI root HUB,
class 9/0, rev 2.00/1.00, addr 1> on usbus3
Jun 15 17:42:22 ibb kernel: ugen4.1: <Intel> at
usbus4
Jun 15 17:42:22 ibb kernel: uhub3: <Intel UHCI root HUB,
class 9/0, rev 1.00/1.00, addr 1> on usbus4
Jun 15 17:42:22 ibb kernel: ugen5.1: <Intel> at
usbus5
Jun 15 17:42:22 ibb kernel: uhub4: <Intel UHCI root HUB,
class 9/0, rev 1.00/1.00, addr 1> on usbus5
Jun 15 17:42:22 ibb kernel: ugen6.1: <Intel> at
usbus6
Jun 15 17:42:22 ibb kernel: uhub5: <Intel UHCI root HUB,
class 9/0, rev 1.00/1.00, addr 1> on usbus6
Jun 15 17:42:22 ibb kernel: ugen7.1: <Intel> at
usbus7
Jun 15 17:42:22 ibb kernel: uhub6: <Intel EHCI root HUB,
class 9/0, rev 2.00/1.00, addr 1> on usbus7
Jun 15 17:42:22 ibb kernel: ugen0.1: <Intel> at
usbus0
Jun 15 17:42:22 ibb kernel: uhub7: <Intel UHCI root HUB,
class 9/0, rev 1.00/1.00, addr 1> on usbus0
Jun 15 17:42:22 ibb kernel: ad4: 238475MB <Hitachi
HTS543225L9A300 FBEOC40F> at ata2-master SATA300
Jun 15 17:42:22 ibb kernel: acpi_tz1: _CRT value is absurd,
ignored (256.0C)
Jun 15 17:42:22 ibb kernel: acd0: DVDR <Optiarc DVD RW
AD-7561S/AH03> at ata3-master SATA150
Jun 15 17:42:22 ibb kernel: SMP: AP CPU #1 Launched!
Jun 15 17:42:22 ibb kernel: WARNING: WITNESS option
enabled, expect reduced performance.
Jun 15 17:42:22 ibb kernel: uhub0: 2 ports with 2
removable, self powered
Jun 15 17:42:22 ibb kernel: uhub1: 2 ports with 2
removable, self powered
Jun 15 17:42:22 ibb kernel: uhub3: 2 ports with 2
removable, self powered
Jun 15 17:42:22 ibb kernel: uhub4: 2 ports with 2
removable, self powered
Jun 15 17:42:22 ibb kernel: uhub5: 2 ports with 2
removable, self powered
Jun 15 17:42:22 ibb kernel: uhub7: 2 ports with 2
removable, self powered
Jun 15 17:42:22 ibb kernel: GEOM: ad4s2: geometry does not
match label (255h,63s != 16h,63s).
Jun 15 17:42:22 ibb kernel: Root mount waiting for: usbus7
usbus3
Jun 15 17:42:22 ibb last message repeated 2 times
Jun 15 17:42:22 ibb kernel: uhub2: 6 ports with 6
removable, self powered
Jun 15 17:42:22 ibb kernel: uhub6: 6 ports with 6
removable, self powered
Jun 15 17:42:22 ibb kernel: ugen3.2: <Chicony
Electronics Co., Ltd.> at usbus3
Jun 15 17:42:22 ibb kernel: Trying to mount root from
ufs:/dev/ad4s2a
Jun 15 17:42:23 ibb kernel: msk0: link state changed to UP
Jun 15 17:42:38 ibb login: ROOT LOGIN (root) ON ttyv0
Jun 15 17:46:53 ibb kernel: lock order reversal:
Jun 15 17:46:53 ibb kernel: 1st 0xd9537bb0 bufwait
(bufwait) @ /usr/src/sys/kern/vfs_bio.c:2558
Jun 15 17:46:53 ibb kernel: 2nd 0xc5efd000 dirhash
(dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:285
Jun 15 17:46:53 ibb kernel: KDB: stack backtrace:
Jun 15 17:46:53 ibb kernel:
db_trace_self_wrapper(c0c5e1c8,e813a87c,c08b15d5,c08a250b,c0c6100b,...)
at db_trace_self_wrapper+0x26
Jun 15 17:46:53 ibb kernel:
kdb_backtrace(c08a250b,c0c6100b,c552acf0,c5530c00,e813a8d8,...)
at kdb_backtrace+0x29
Jun 15 17:46:53 ibb kernel:
_witness_debugger(c0c6100b,c5efd000,c0c810bd,c5530c00,c0c80d56,...)
at _witness_debugger+0x25
Jun 15 17:46:53 ibb kernel:
witness_checkorder(c5efd000,9,c0c80d56,11d,0,...) at
witness_checkorder+0x839
Jun 15 17:46:53 ibb kernel:
_sx_xlock(c5efd000,0,c0c80d56,11d,c6015c3c,...) at
_sx_xlock+0x85
Jun 15 17:46:53 ibb kernel:
ufsdirhash_acquire(d9537b50,da6db800,200,da6db81c,e813a9a8,...)
at ufsdirhash_acquire+0x35
Jun 15 17:46:53 ibb kernel:
ufsdirhash_add(c6015c3c,e813aa20,81c,e813a994,e813a998,...)
at ufsdirhash_add+0x13
Jun 15 17:46:53 ibb kernel:
ufs_direnter(c5ff596c,c6139218,e813aa20,e813ac04,d956c200,...)
at ufs_direnter+0x729
Jun 15 17:46:53 ibb kernel:
ufs_mkdir(e813ac28,ead,0,0,e813ab70,...) at ufs_mkdir+0x897
Jun 15 17:46:53 ibb kernel:
VOP_MKDIR_APV(c0d5eec0,e813ac28,e813ac04,e813ab70,0,...) at
VOP_MKDIR_APV+0xa5
Jun 15 17:46:53 ibb kernel:
kern_mkdirat(c5cda480,ffffff9c,28528f80,0,1ed,...) at
kern_mkdirat+0x268
Jun 15 17:46:53 ibb kernel:
kern_mkdir(c5cda480,28528f80,0,1ed,e813ad2c,...) at
kern_mkdir+0x2e
Jun 15 17:46:53 ibb kernel:
mkdir(c5cda480,e813acf8,8,c0c5e284,c0d3f240,...) at
mkdir+0x29
Jun 15 17:46:53 ibb kernel: syscall(e813ad38) at
syscall+0x2a3
Jun 15 17:46:53 ibb kernel: Xint0x80_syscall() at
Xint0x80_syscall+0x20
Jun 15 17:46:53 ibb kernel: --- syscall (136, FreeBSD
ELF32, mkdir), eip = 0x283063c3, esp = 0xbf4f9d1c, ebp =
0xbf4f9d48 ---

Mail me, if I can help with anything else...


More precisely, I see this LOR when I try to configure custom kernel with:
#config MYKERN





_______________________________________________
freebsd-current@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@xxxxxxxxxxx"

Relevant Pages