Re: memory fragmentation with malloc

On Wed, 1 Aug 2007 11:57:28 +0200, "G. Maly" <gemal@xxxxxxxxxxxxxx> wrote:
seems to be optimized in terms of speed. umem will assume that the
999999 freed blocks will be used again soon and cache them, but not
give them back to the OS. isn't it?

libumem is a bit more complex than the description above.

There is an integrated 'cache reaping' mechanism, which runs either on a
time-based interval or when the heap needs to be grown. This cache
reaping mechanism tries to free up enough resources to satisfy
allocation requests from already freed (but cached) objects.

So libumem won't give freed (but cached) areas "back to the OS", but it
will happily reuse them to satisfy future allocation requests blazingly
fast.

It will not reuse them happily if the application will NOT need such
large amount of memory any more or at least in the next time.
Assume the application which runs
1) short phases with large memory consumption and
2) long phases with small memory consumption.

It would be preferable if VirtMemory would be reduced during phase 2.
The question was which memory allocator does support it?

Since cache reaping is not based only on memory demand/pressure but it
is also time-triggered after a configurable period of seconds, libumem
*will* return unused memory from its cache after the time-based reap
happens.

Doesn't that cover the (1) and (2) phases described above?

.

Relevant Pages

  • Re: Cached memory never gets released
    ... Stock linux 2.4.26 kernel. ... Due to flash bug 3M of memory gets lost due to font memory getting lost ... The output of "free" cache number steadily grows. ... longer to exhaust all of system memory with the cache. ...
    (Linux-Kernel)
  • Re: Problem: Creating a raw binary string
    ... > While its true that a 64-bit cpu will move twice the data per instruction it ... > Memory bus width plays an important role here and unless it too is widened / ... You are forgetting the two levels of cache in the processor. ... The memory chips are addressed in Row col fashion. ...
    (alt.comp.lang.borland-delphi)
  • Re: Is Greenspun enough?
    ... Most OSes memory map executables directly from the file system so code doesn't pollute the file cache or swap space. ...
    (comp.lang.lisp)
  • Re: Superstitious learning in Computer Architecture
    ... Without a LOT of logic or some other better approach, re-executing the instructions requires re-decoding and it ties up the cache memory bus transferring more data as instructions than the instructions are working on. ... The concept of cache is fundamentally flawed in that it STILL restricts access to one word per clock cycle, when a single modern ALU can easily use 5 plus whatever is eaten up with instruction accesses. ... The size of an optimizing compiler is proportional to the SQUARE of the size of the language times the SQUARE of the complexity of the machine - because all interactions must be considered. ...
    (comp.arch.arithmetic)
  • Re: High-bandwidth computing interest group
    ... sequential access patterns, brute force - neither of us consider that interesting ... Perhaps we should lose the cache line orientation - transferring data bytes that aren't needed. ... Particularly if it has scatter/gather vector instructions like Larrabee, or if it is a CIMT coherent threaded architecture like the GPUs. ... As I have discussed in this newsgroup before, this allows us to have writeback caches where multiple processors can write to the same memory location simultaneously. ...
    (comp.arch)