Re: Automatic reboot doesn't reboot
- From: Jeremy Chadwick <freebsd@xxxxxxxxxxxxxxxx>
- Date: Tue, 3 May 2011 05:20:52 -0700
On Tue, May 03, 2011 at 12:08:54PM +0200, Olaf Seibert wrote:
On Tue 03 May 2011 at 02:21:13 -0700, Jeremy Chadwick wrote:
There are two things you might try fiddling with. These are sysctls so
you can try them on the fly:
Thanks. For now I've set the second to 1 and we'll see if that affects
Check out the thread Peter Jeremy provided. This is a near-sure...
indicator of ZFS ARC exhaustion, and you seem to know of that. What's
very interesting to me is this part of your mail:
Is this box running i386 or amd64? If amd64, I can't explain why your
It's amd64. I double-checked just one, you never know what stupid
mistakes one might make :-)
/boot/loader.conf settings aren't taking -- they should be for sure.
Maybe provide us a full dmesg and XXX out things you consider
sensitive. If i386, I'm not too surprised that some automatic defaults
get chosen instead of what you ask.
Based on one of your mails where setting vm.kmem_size to twice the real
RAM size had adverse effects, I've taken the setting out to see if that
improves matters. I'll have to wait until the next crash (or opportunity
to reboot without too much disturbance) to see the effect.
The ill-effects are a result of an underlying change that I had
forgotten about but others remembered -- vm.kmem_size_scale used to be
set to something like "2" by default, but it was changed to "1" prior to
So basically here's the current situation and how all of our 8.2-STABLE
machines are tuned for ARC: we only set one single tunable for ARC
"management": vfs.zfs.arc_max. We don't touch vm.kmem_size.
Here's what we have literally in our /boot/loader.conf:
# Limit ZFS ARC maximum.
# NOTE #1: In 8.2-RELEASE and onward, vm.kmem_size_scale defaults to 1,
# which means vm.kmem_size should match the amount of RAM installed
# in the system. If using an earlier FreeBSD release, be sure to set
# vm.kmem_size manually to the amount of RAM you have.
# NOTE #2: Do not set vm.kmem_size to 2x that of physical RAM, otherwise
# vfs.zfs.arc_max effectively becomes halved.
The value specified here (6144MBytes) is for a machine with 8GB of RAM.
Keep in mind that there is evidence that kmap/kmem exhaustion can still
happen even if you tune the ARC like this. Apparently memory
fragmentation plays a role, and there's some overhead as well, so
calculating a 100% stable value is a little difficult. I can point you
to that (very recent, as in last month) thread if you'd like.
To be on the safe side, pick something that's small at first, then work
your way up. You'll need probably 1+ weeks of heavy ZFS I/O between
tests (e.g. don't change the tunable, reboot, then 4 hours later declare
the new (larger) value as stable).
So for example on an 8GB RAM machine, I might recommend starting with
vfs.zfs.arc_max="4096M" and let that run for a while. If you find your
"Wired" value in top(1) remains fairly constant after a week or so of
heavy I/O, consider bumping up the value a bit more (say 4608M).
Sorry to make this long-winded; bad habit of mine that I've never
managed to break.
| Jeremy Chadwick jdc@xxxxxxxxxxxx |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP 4BD6C0CB |
freebsd-stable@xxxxxxxxxxx mailing list
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@xxxxxxxxxxx"
- Re: Automatic reboot doesn't reboot
- From: Olaf Seibert
- Re: Automatic reboot doesn't reboot
- Prev by Date: Re: RELENG_8 pf stack issue (state count spiraling out of control)
- Next by Date: Re: Automatic reboot doesn't reboot
- Previous by thread: Re: Automatic reboot doesn't reboot
- Next by thread: Re: Automatic reboot doesn't reboot