Re: Automatic reboot doesn't reboot

On Tue, May 03, 2011 at 12:08:54PM +0200, Olaf Seibert wrote:
On Tue 03 May 2011 at 02:21:13 -0700, Jeremy Chadwick wrote:
There are two things you might try fiddling with. These are sysctls so
you can try them on the fly:

hw.acpi.disable_on_reboot
hw.acpi.handle_reboot

Thanks. For now I've set the second to 1 and we'll see if that affects
matters.

Check out the thread Peter Jeremy provided. This is a near-sure
indicator of ZFS ARC exhaustion, and you seem to know of that. What's
very interesting to me is this part of your mail:
...

Is this box running i386 or amd64? If amd64, I can't explain why your

It's amd64. I double-checked just one, you never know what stupid
mistakes one might make :-)

/boot/loader.conf settings aren't taking -- they should be for sure.
Maybe provide us a full dmesg and XXX out things you consider
sensitive. If i386, I'm not too surprised that some automatic defaults
get chosen instead of what you ask.

Based on one of your mails where setting vm.kmem_size to twice the real
RAM size had adverse effects, I've taken the setting out to see if that
improves matters. I'll have to wait until the next crash (or opportunity
to reboot without too much disturbance) to see the effect.

The ill-effects are a result of an underlying change that I had
forgotten about but others remembered -- vm.kmem_size_scale used to be
set to something like "2" by default, but it was changed to "1" prior to
8.2-RELEASE.

So basically here's the current situation and how all of our 8.2-STABLE
machines are tuned for ARC: we only set one single tunable for ARC
"management": vfs.zfs.arc_max. We don't touch vm.kmem_size.

Here's what we have literally in our /boot/loader.conf:

# Limit ZFS ARC maximum.
# NOTE #1: In 8.2-RELEASE and onward, vm.kmem_size_scale defaults to 1,
# which means vm.kmem_size should match the amount of RAM installed
# in the system. If using an earlier FreeBSD release, be sure to set
# vm.kmem_size manually to the amount of RAM you have.
# NOTE #2: Do not set vm.kmem_size to 2x that of physical RAM, otherwise
# vfs.zfs.arc_max effectively becomes halved.
# http://lists.freebsd.org/pipermail/freebsd-fs/2011-March/010875.html
vfs.zfs.arc_max="6144M"

The value specified here (6144MBytes) is for a machine with 8GB of RAM.

Keep in mind that there is evidence that kmap/kmem exhaustion can still
happen even if you tune the ARC like this. Apparently memory
fragmentation plays a role, and there's some overhead as well, so
calculating a 100% stable value is a little difficult. I can point you
to that (very recent, as in last month) thread if you'd like.

To be on the safe side, pick something that's small at first, then work
your way up. You'll need probably 1+ weeks of heavy ZFS I/O between
tests (e.g. don't change the tunable, reboot, then 4 hours later declare
the new (larger) value as stable).

So for example on an 8GB RAM machine, I might recommend starting with
vfs.zfs.arc_max="4096M" and let that run for a while. If you find your
"Wired" value in top(1) remains fairly constant after a week or so of
heavy I/O, consider bumping up the value a bit more (say 4608M).

Sorry to make this long-winded; bad habit of mine that I've never
managed to break.

--
| Jeremy Chadwick jdc@xxxxxxxxxxxx |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP 4BD6C0CB |

_______________________________________________
freebsd-stable@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: Setting "zfs_arc_max" value in FreeBSD 8.
    ... Now, throw in an SSD for L2ARC, more RAM for ARC (and L2ARC ... OpenSolaris on the same hardware. ...
    (freebsd-hackers)
  • Re: Rasberry Pi dishonesty not thought through properly
    ... Raspberry Pi couldn't look more different from an Arc. ... CPU performance with a highly-efficient, ... backed up by 512Mb of high performance RAM. ... RISC OS 5, with licences from Castle and RISC OS Ltd. ...
    (uk.comp.os.linux)
  • Re: Setting "zfs_arc_max" value in FreeBSD 8.
    ... Anyone know if it is adjustable on a system with 1024MB of ram? ... real memory = 1072107520 ... It needs to be a lot less than kmem_max (the arc is allocated from ... the kernel also need kmem, you need to limit the arc size). ...
    (freebsd-hackers)
  • Re: ZFS vs OSX Time Machine
    ... The difference is probably due to fragmentation (most of ARC ... It did have 4GB of RAM but that was fairly terrible, 8GB is a lot better though:) ... for Genesis Software - http://www.gsoft.com.au ...
    (freebsd-stable)
  • Re: ZFS ARC being limited below what is defined in /boot/loader.conf
    ... If you want to bump ARC size, you do need to bump vm.kmem_size. ... However, after rebooting, querying the sysctl gives me this: ... of RAM. ... amd64. ...
    (freebsd-stable)