Re: Proposed addition of malloc_size_np()


:...
:have to store the precise allocation request size; it can instead round
:the request size up internally, then treat the allocation as being of
:this rounded up size. By vaguely specifying the return value of
:malloc_size_np(), we grant the malloc implementation freedom as to
:whether the size is precisely what was specified during allocation, or
:some rounded up value.
:
:I had no intention of suggesting that malloc_size_np() should extend
:existing allocations, nor change the return value depending on the
:current state of memory following the allocation pointed to by ptr.

I'm not sure what benefit this would have, though. When a program
calls malloc it generally does so with a good idea as to the number
of bytes it wants to allocate. malloc implementations have always
been free to return more bytes then were requested. But why would
any program want to complicate its life and use an API where the
potential additional space (which is extremely algorithm-dependant)
becomes visible to the program? I'm trying to imagine the number of
mistakes programmers would make, with the resulting bugs winding up in
the program(s), that an integrated implementation would cause.

An API like malloc_size_np() would have a very limited usefullness,
not to mention being inflexible. Programs are far more likely to
want to resize an allocation then to use whatever extra slop space the
malloc implementation might happened to have included in the original
allocation.

A reallocation API like realloc_try() makes a lot more sense. Not
only does it cover all aspects of malloc_size_np(), it also gives
the allocator implementation the flexibility to actually truncate or
extend an allocation if that happens to be supported by the allocator,
or not if it doesn't.

:This actually does make some assumptions about the way memory is managed
:by malloc. It would not be useful in the context of phkmalloc or
:jemalloc to truncate an allocation via realloc_try(), because each page
:of memory contains allocations of only one size class. Thus, there is
:no way to re-use the trailing space (exception: multi-page allocations).
: This would potentially be useful for dlmalloc (or ptmalloc) though.
:
:Thanks,
:Jason

No, it actually doesn't. Or I should say, uf a program is going to
actually try to USE the potential extra space for something, something
like realloc_try() makes no more assumptions about the malloc
implementation than malloc_size_np(). The implementation is totally
free to limit the functionality of realloc_try() to the same slop
space (if any) that malloc_size_np() returns.

The difference is that realloc_try() is far closer to what a programmer
would likely need. Even for things like slab allocators which have no
ability to extend or truncate small allocations beyond the chunk size
of the slab they reside in, many malloc implementations still use
a different mechanism for larger multi-page allocations and in those
cases the implementation might actually be able to do something with
the original allocation.

The only real situation that I can think of where a program might want
to have the option of (optimally) using more or less space then was
originally requested is the situation where a large chunk of memory,
such as 32KB, needs to be extended or truncated. For example, a
preprocessor or parser (generating post-parsed output) or linker
(relocation arrays), or any other program of that nature could benefit.

I'm still wondering who in their right mind would actually malloc X bytes
and then use an API call to obtain and actually use the (dynamic,
ephermal, and completely non-deterministic) Y bytes that the underlying
implementation reserved, verses simply allocating Y bytes in the first
place. It sounds like a recipe for disaster to me. At least with a
reallocation API such as realloc_try() the caller can specify a
requirement that the malloc implementation is either able to meet or not
able to meet.

-Matt
Matthew Dillon
<dillon@xxxxxxxxxxxxx>
_______________________________________________
freebsd-arch@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: How to know the memory pointed by a ptr is freed?
    ... One way in which this could cause your test to fail is if the second ... time space is malloced for q it uses the space immediately before where ... p was allocated as part of its allocation. ... reasonable thing for the malloc implementation to do. ...
    (comp.lang.c)
  • Re: Implicit int
    ... and had to change my own malloc implementation to "touch" the ... operating system to report success on storage allocation when the ... the OS doesn't necessarily kill the program that attempted ... the allocation; it might kill some other process. ...
    (comp.std.c)
  • Proposed addition of malloc_size_np()
    ... It is up to the implementation whether the value returned by malloc_size_npdiffers from the size specified during allocation, but the return value must be consistent across multiple calls for the same allocation. ... This can be useful for a variety of purposes, including introspection and putting wrappers around the system malloc implementation. ... OS X provides malloc_sizeand malloc_good_size, which return the precise allocation request size and the usable size of the allocation, respectively. ... By providing malloc_size_np, we provide an analogue to these APIs, without placing undue constraints on future malloc implementations. ...
    (freebsd-arch)
  • Re: Proposed addition of malloc_size_np()
    ... This function would return the usable size of the allocation pointed to ... malloc implementation. ... without placing undue constraints on future ... the proposed semantics differ from those of OS X's ...
    (freebsd-arch)
  • [PATCH -mm 03/14] bootmem: add documentation to API functions
    ... pfn where the bitmap for this node is to be placed ... size of the request in bytes ... * fall back to memory below @goal. ... * Allocation may happen on any node in the system. ...
    (Linux-Kernel)