Process can't allocate more memory, even though there is more

From: Schoep, Grant _at_ STORM (_at_)
Date: 06/12/03

  • Next message: Popovich, Tom: "If you are running release 4.0g can you run the following command s against your /usr/bin/passwd" 
    Date: Wed, 11 Jun 2003 20:23:08 -0400
To: "'tru64-unix-managers@ornl.gov.'" <tru64-unix-managers@ornl.gov>

    Having an issue with a Tru64 4.0f machine(ok all of them).

    We have a process that is growing fairly steadily. It really uses a lot of
    memory. Under Solaris it levels off around 700 megs. But on 4.0f, the app
    errors out around 300 meg as a malloc call returns 0(out of memory).

    I'm guessing this is one of those many really annoying kernel limit settings
    that is limiting our process's memory size. I thought I figured it out, but
    am having a bit of trouble.

    Looking at the Sys Config and Tuning guide, I guessed it was
    per-proc-data-size. So I kicked it up 5 times from its default. I just
    rebooted after this(this doesn't need a kernel rebuild does it?)

    sysconfig -q proc now reports the new number.

    However, app still errors out at 300 megs.

    So, I ran trusty old sys_check -perf and made all its recommended
    changes(and backed out the previous one as it didn't change anything).

    No luck... still errors out.

    Some info about the machine. It only has 1 gig. But on our normal machines
    that all have 2-4 gigs of RAM they do the same thing.

    Its a PW 1000XP

    It has 3 gigs of swap space.

    It is running 4.0f( upgrading to any patch level from this is not currently
    an option...)

    I reproduced this issue quickly by making some code that basically just runs
    in a for loop and mallocs tons of memory until malloc returns 0. It does
    this at about 300 megs.

    So more machine info below. If anyone can point me something to try that
    would be great.
    Thanks
    -grant

    As it may help, I also am pasting in the following info. Note that all our
    processes are running after a "unlimit(csh)" command. Output of the limit
    command before and after.

    grant maroon>27% limit
    cputime unlimited
    filesize unlimited
    datasize 131072 kbytes
    stacksize 2048 kbytes
    coredumpsize unlimited
    memoryuse 1020384 kbytes
    descriptors 4096 files
    addressspace 1048576 kbytes
    grant@maroon>28% unlimit
    grant@maroon>29% limit
    cputime unlimited
    filesize unlimited
    datasize 1048576 kbytes
    stacksize 32768 kbytes
    coredumpsize unlimited
    memoryuse 1019856 kbytes
    descriptors 4096 files
    addressspace 1048576 kbytes
    grant@maroon>30% ulimit

    I also did a "sysconfig -q" on vm and proc as I figured those are most
    applicable. The output of these was AFTER I setup the recommended settings
    from syscheck -perf.
    rant@maroon>27% limit
    cputime unlimited
    filesize unlimited
    datasize 131072 kbytes
    stacksize 2048 kbytes
    coredumpsize unlimited
    memoryuse 1020384 kbytes
    descriptors 4096 files
    addressspace 1048576 kbytes
    grant@maroon>28% unlimit
    grant@maroon>29% limit
    cputime unlimited
    filesize unlimited
    datasize 1048576 kbytes
    stacksize 32768 kbytes
    coredumpsize unlimited
    memoryuse 1019856 kbytes
    descriptors 4096 files
    addressspace 1048576 kbytes
    grant@maroon>30% ulimit

    root@maroon>1% sysconfig -q proc
    proc:
    max-proc-per-user = 384
    max-threads-per-user = 384
    per-proc-stack-size = 2097152
    max-per-proc-stack-size = 33554432
    per-proc-data-size = 134217728
    max-per-proc-data-size = 1073741824
    max-per-proc-address-space = 1073741824
    per-proc-address-space = 1073741824
    executable_stack = 0
    autonice = 0
    autonice-time = 600
    autonice-penalty = 4
    open-max-soft = 4096
    open-max-hard = 4096
    ncallout_alloc_size = 8192
    round-robin-switch-rate = 0
    round_robin_switch_rate = 0
    sched-min-idle = 0
    sched_min_idle = 0
    give-boost = 1
    give_boost = 1
    maxusers = 512
    task-max = 4117
    thread-max = 8232
    num-wait-queues = 512
    num-timeout-hash-queues = 512
    enable_extended_uids = 0
    enhanced-core-name = 1
    enhanced-core-max-versions = 3

    vm:
    ubc-minpercent = 10
    ubc-maxpercent = 100
    ubc-borrowpercent = 20
    ubc-maxdirtywrites = 5
    vm-max-wrpgio-kluster = 32768
    vm-max-rdpgio-kluster = 16384
    vm-cowfaults = 4
    vm-mapentries = 16384
    vm-maxvas = 1073741824
    vm-maxwire = 16777216
    vm-heappercent = 7
    vm-vpagemax = 131072
    vm-segmentation = 1
    vm-ubcpagesteal = 24
    vm-ubcdirtypercent = 10
    vm-ubcseqstartpercent = 40
    vm-ubcseqpercent = 10
    vm-csubmapsize = 1048576
    vm-ubcbuffers = 256
    vm-syncswapbuffers = 128
    vm-asyncswapbuffers = 4
    vm-clustermap = 1048576
    vm-clustersize = 65536
    vm-zone_size = 0
    vm-kentry_zone_size = 16777216
    vm-syswiredpercent = 80
    vm-inswappedmin = 1
    vm-page-free-target = 256
    vm-page-free-swap = 138
    vm-page-free-hardswap = 4096
    vm-page-free-min = 20
    vm-page-free-reserved = 10
    vm-page-free-optimal = 138
    vm-page-prewrite-target = 512
    dump-user-pte-pages = 0
    kernel-stack-guard-pages = 1
    vm-min-kernel-address = 18446744071562067968
    malloc-percpu-cache = 1
    vm-aggressive-swap = 0
    vm_page_color_private = 0
    vm-map-index-count = 64
    vm-map-index-rebalance = 128
    vm-map-index-enabled = 1
    vm-map-index-hiwat = 4
    vm-map-index-lowat = 2
    new-wire-method = 1
    vm-segment-cache-max = 50
    vm-page-lock-count = 0
    gh-chunks = 0
    gh-min-seg-size = 8388608
    gh-fail-if-no-mem = 1
    private-text = 0

  • Next message: Popovich, Tom: "If you are running release 4.0g can you run the following command s against your /usr/bin/passwd"

    Relevant Pages

    • Tru64 issues with Infinite limits
      ... An automated test in my nightly build was failing due to Out of Memory ... Cannot allocate block 146 ...
      (comp.unix.tru64)
    • SUMMARY: Process cant allocate more memory, even though there is more
      ... "unlimit" csh command. ... Process can't allocate more memory, ... memoryuse 1020384 kbytes ...
      (Tru64-UNIX-Managers)
    • Re: PDP11 Memory Mangement
      ... The first thing to understand is that for the PDP-11, ... or at least the systems which were produced when memory ... no map registers for I/O devices was that the maximum memory was 256 KBytes ... >>segments became larger than 64K, was there a restriction that executables ...
      (comp.sys.dec)
    • ulimit and malloc
      ... I'm running Mandrake 7.2 and I'm trying to use the ulimit command: ... max locked memory (kbytes) 1024 ...
      (comp.os.linux.security)
    • ulimit and malloc
      ... I'm running Mandrake 7.2 and I'm trying to use the ulimit command: ... max locked memory (kbytes) 1024 ...
      (comp.os.linux.security)