Re: very high memory usage in -current

Stefan Ehmann wrote:
[huge jpeg/png images cause memory bloat with konqueror and gtk apps, on -CURRENT.]

I don't really know what's causing this problem. Maybe it's related to jemalloc, but I'd be surprised if no one else has noticed this before.

size/res values reported by top

6.2 desktop pc with nvidia binary driver
konqueror 40M/30M

CURRENT notebook with the i810 driver
konqueror 290M/250M

Any clues or more information needed?

If you think jemalloc is involved, the easiest way to check is by
reverting src/lib/libc/stdlib/malloc.c to revision 1.92, which is
phkmalloc. If this substantially changes memory usage, then there are
further diagnostics that can be used to help understand the issue.

-----

If jemalloc is involved, here's what could cause such behavior. First,
you would have to be running on a 32-bit platform, so that sbrk() is in
use (rather than pure mmap() as for the 64-bit platforms).

1) The application would allocate some large chunks of memory for image
processing. These data would be allocated in the data segment.

2) The application would allocate some additional long-lived object for
which no existing memory region could be recycled. This would cause the
object to be allocated in the data segment, at the end of the sbrk()ed
memory.

3) The application would free most/all of the memory that was used for
image processing. Unfortunately, due to the long-lived object from step
(2), this would be in the middle of the sbrk()ed memory, and so malloc
would have to hold onto the memory for possible future use, unable to
return it to the kernel.

The right solution to the problem is to get rid of the hard line between
data segment and heap, and use the address space that is currently
reserved for sbrk() as the last resort for mmap() requests, thus
allowing the data segment and heap to gracefully grow toward each other
until memory is exhausted. We really don't need the data segment
anymore except for legacy support of applications that do their own
memory management via sbrk(). However, when I talked with the VM
wizards, I was sufficiently intimidated by the difficulty of removing
the hard-coded data segment size that I did not pursue it any further.

Personally, in the absence of a dynamic boundary between the data
segment and the heap, I would be quite happy to completely disable
sbrk() support in jemalloc, and let those who really need that last 512
MB of address space adjust resource limits for their applications as
necessary. In practice, I expect this would cause people far less
trouble than does the current state of affairs.

Jason

_______________________________________________
freebsd-current@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: Static constructors/destructors
    ... Object instances go to the data segment and anything in data segment must be freed as soon as process frees an object pointer. ... If a static field is subject to change, I am not sure, but the pointer to it should be in text segment and the memory should be allocated on stack/heap depending on it's type, even in that case the pointer can not be dereferenced and so, the memory used by static fields will never be deleted in process lifetime. ... it doesn't quite make sense to match a static constructor with a single destructor. ...
    (microsoft.public.dotnet.languages.vc)
  • Re: Kind of new: function implementation questions, MASM
    ... In 32-bit programming, you should simply ignore these concepts. ... A stack is just a piece of memory. ... >indicate it's a register definition. ... >the data segment to be a block of memory that I can access. ...
    (comp.lang.asm.x86)
  • Complete list of process attributes?
    ... memory, shared segments ... memory, shared segments (SysV) ... memory, data segment, arguments ... semaphores, memory ...
    (comp.unix.programmer)
  • Re: Doubts on C
    ... >> slot i,e total memory of the Heap I can allocate in one slot. ... data segment, and the code segment, Form the heap only all the ...
    (comp.lang.c)
  • Re: doubt in protected mode
    ... I have doubt about the memory manager. ... When i compile any asm ... zero-out CS/DS in 32-bit Protected Mode, ... protected or how to know whether the content of code and data segment ...
    (alt.lang.asm)