Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

On Thu, Sep 11, 2008 at 11:20 AM, Jeremy Chadwick <koitsu@xxxxxxxxxxx> wrote:
On Thu, Sep 11, 2008 at 10:38:36AM +0200, Michael Grant wrote:
My box crashed again:

panic: kmem_malloc(4096): kmem_map too small: 1073741824 total allocated
cpuid = 0
Uptime: 33d11h12m58s
Dumping 3327 MB (2 chunks)
chunk 0: 1MB (151 pages) ... ok
chunk 1: 3327MB (851568 pages) <---hung here

Still no valid dump.

There is 4gig of physical memory in the machine.

In /boot/loader.conf, I currently have the following:

vm.kmem_size=1G
vm.kmem_size_max=1G
vm.kmem_size_scale=2

and in my kernel conf file I have:

options KVA_PAGES=512

It stayed up for 33 days this time. Is there anything else I can do?

First and foremost: are you using ZFS on this machine? If so, there are
many tunables you can apply to try and limit this; I'm willing to bet
it's ARC which is doing it. See below.

In general, it appears that you need to increase the maximum range of
kmem. The kernel attempted to utilise more than 1GB, and your limit is
1G. My machines running RELENG_7 on amd64, with only 2GB of RAM
installed, use the following tunables in loader.conf:

vm.kmem_size="1536M"
vm.kmem_size_max="1536M"

If ZFS is in use, I recommend these as well:

vfs.zfs.arc_min="16M"
vfs.zfs.arc_max="64M"
vfs.zfs.prefetch_disable="1"

Do not increase kmem_size any larger than 1.5GB; the amount of RAM you
have in the machine, with regards to RELENG_7, will not help. This is a
known limitation which has been fixed in HEAD/CURRENT (where the limit
has been increased to 512GB). See the "Kernel" section below; you'll
see the applicable item.

http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues

Your only solution may be to run HEAD/CURRENT.

I am not running ZFS. My file systems are ufs.

This feels like some sort of memory leak in the kernel. Giving it
more and more memory just seems to delay the crash. Are you saying
the crash is fixed in HEAD/CURRENT?

I'm running 6.3 by the way.

I have put your changes into my loader.conf, we'll see how long it
goes this time. I'm not qute in position to update everything to 7.x
at the moment.

Michael Grant
_______________________________________________
freebsd-stable@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: ZFS boot
    ... prefetch for zfs. ... I was somewhat confused by the suggestions on the wiki. ... amount of memory which can be used by the kernel for allocation. ... ZFS is a memory hog, ...
    (freebsd-hackers)
  • Re: BETA3 crash (zfs related ?)
    ... Although this is a guess, I'm relatively confident that you are having the same problem I had, because my impression is that right this moment anyone who uses ZFS intensively with a current kernel will meet this problem for sure. ... it seems that the ARC was using too much memory. ... The ARC also sets a limit on how much of that memory it will use ...
    (freebsd-current)
  • Re: kmem_map too small
    ... of memory allocated to the kernel). ... ZFS on my desktop at work: two weeks for testing is very large amount of ... but I can't say that any other test will not crash it. ...
    (freebsd-stable)
  • Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load
    ... are you using ZFS on this machine? ... many tunables you can apply to try and limit this; ... The kernel attempted to utilise more than 1GB, ... Your only solution may be to run HEAD/CURRENT. ...
    (freebsd-stable)
  • 2.6.33 dies on modprobe
    ... The kernel starts up fine and mounts the root-filesystem, ... Freeing unused kernel memory: 408k freed ... # CPUFreq processor drivers ...
    (Linux-Kernel)