Re: FreeBSD 5.1-R kernel panic

From: Terry Lambert (tlambert2_at_mindspring.com)
Date: 07/25/03

  • Next message: Terry Lambert: "Re: fuword(), suword(), etc."
    Date: Thu, 24 Jul 2003 23:27:31 -0700
    To: Stephane Raimbault <segr@hotmail.com>
    
    

    Stephane Raimbault wrote:
    > Well I went to go change my /boot/loader.conf options to reflect the
    > following:
    >
    > kern.vm.kmem.size="350000"

    Assuming this is in pages, it is 1/3 of the total physical RAM in the
    system. This is way too large, unless you have recompiled your kernel
    to have 3G KVA vs. 1G UVA, instead of the default. Kernel recompilation
    is required because the load address is a fixed virtual address based on
    the KVA size, subtracted from the top of memory. This is so that user
    programs do not have to be recompiled when you move the ratio of UVA to
    KVA around (with the exception of the Linux emulator, if you are using
    Linux threads, since it seems to want its threads mailboxes at a known
    location in memory).

    > kern.ipc.nmbclusters="8192"

    This is large, as well.

    You need to turn off automatic tuning, and act like you are on a
    memory budget equal to the amount of KVA space in the system,
    which you can use for different things, but not at the same time.

    If you leave the system at the defaults, it will (mostly) be able
    to keep itself under the budget ceiling for an assumed 2G KVA
    space, but as soon as you start tuning some things up, you will
    very quickly blow your budget, since you tuning something you want
    to use up will not necessarily tune something you aren't going to
    use down, to avoid blowing your total budget.

    > and enabled "options DDB" in my kernel.
    >
    > Unfortunately, I ran into a problem on the reboot, the SMP kernel would fail
    > to load due to some of my /boot/loader.conf changes.... perhaps I
    > miss-understood something in your instructions... anyhow, this is what was
    > displayed on the console when I set those above settings in the
    > /boot/loader.conf... and since I had the debugger running, I did a trace and
    > included in the paste below... is that the kind of stuff you will want if
    > the box crashes again as originally mentioned in this thread.

    This particular panic is an initialization of a region with a
    header structure, where the memory allocation has failed because
    you have already blown your budget, and so the allocation returns
    NULL.

    This happens early enough in the system startup that there is no
    error checking to ensure that the allocation has succeeded. But
    even if you added error checking in all these places, the best the
    system is going to be able to do is pnaic with a slightly more
    informative message.

    When it goes to fill in this structure, it tries to fill in a
    field containing something 8 bytes into the structure, and
    explodes. If you look at the function where the traceback says
    it crashed, this should be visibly obvious to you.

    A good rule of thumb for tuning is to start with the autotuned
    defaults (though they can screw you on occasion, since RAM is
    installed in discrete amounts, and the autotuning tends to use
    a continuous function of the amount of physical RAM, so you get
    a "stair function", and not all possible steps in the stair have
    actually been tested with all possible resource allocations having
    been made, to see if a problem occurs).

    Once you know those, you hard-code them so that they are no longer
    autotuned.

    Then you tweak the resource allocations for resources you don't
    care about using down.

    Then you tweak what you do care about using up, until you crash.

    Then you back off on your last tweak to give you some safety
    margin.

    This will get you within 85%-90% of where you can get without
    modifying code.

    The only other alternative is to know where every byte of memory
    is going.

    -- Terry
    _______________________________________________
    freebsd-current@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-current
    To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"


  • Next message: Terry Lambert: "Re: fuword(), suword(), etc."

    Relevant Pages

    • Re: How to release heap memory that is marked as free
      ... As I said, fragmentation is a very serious problem, and one of the most serious problems ... my allocator was accused of using massive amounts of memory. ... I'm going to have to re-think the memory allocation that I'm ... process's 'working set'. ...
      (microsoft.public.vc.mfc)
    • Re: [PATCH 00/28] Swap over NFS -v16
      ... memory they can consume. ... So we need the extra (skb) ... included in the reserve? ... if the allocation had to dip into emergency reserves, ...
      (Linux-Kernel)
    • Re: Memory leak with CAsyncSocket::Create
      ... read my essay on how storage allocators work. ... Create method is consuming system memory that is not released back to ... The memory consumption is either shown as "Mem Usage" on the Task ... many levels of allocation going ...
      (microsoft.public.vc.mfc)
    • Re: OT: C++ overloading operators
      ... dynamic allocation, no matter how many "clever tricks" are used... ... though there's enough memory in the system, ... all these "flexible data types" map into CPU command ... The computing environment is completely ...
      (comp.dsp)
    • Re: HLA Lib
      ... All memory allocation is freed up when the process quits. ... reduce need to resize blocks for 98% string operations. ... HLA strings already consume. ...
      (alt.lang.asm)