Re: Proposed addition of malloc_size_np()

On Monday 27 March 2006 11:34, Jason Evans wrote:
John Baldwin wrote:
On Sunday 26 March 2006 13:04, Jason Evans wrote:

Robert Watson wrote:

I wonder if the intuitive objection people are raising is actually with
the name. Since malloc_size() is defined on at least one platform to
return the requested size, maybe a name like malloc_allocated_size() (or
something a bit more compact) would help avoid that confusion, and make
it clear that the consumer is getting back a commitment and not a hint
for future realloc(), etc.

Maybe you're right. We could just call it malloc_usable_size() and be
compatible with Linux.

It would help to know why such a function would be useful. :) Do you have
a specific use-case? If the purpose is for a program to see that it really
as Y >= X bytes when it did malloc(X) so that the program can use Y bytes,
that would seem to only be a source of bugs and complexity. If the program
wants Y bytes, it should malloc(Y). Many folks in the thread seem to think
that this function would be used for a poor-man's realloc() wrapper or
something, and I think such uses would be short-sighted at best. If there
are other uses such as for having a debug malloc wrap the real one, then
that might justify the API, but it is unclear what a useful use of this API
would be.

I can think of a few straightforward uses:

1) Suppose that we are about to write to a string that we know is
malloc()ed, and we want to be sure that we aren't overflowing the
allocated buffer. We can add an assertion that the buffer is indeed
large enough to contain what is about to be written to it.

In that case though you really want to know the size you allocated,
not necessarily the backing-store size, esp. if you are in a library
since your caller may (correctly) assume that if it allocates X
bytes for a string the library will only write up to X bytes. So, I
think in that case you want the requested size, not the backing store
size.

2) Suppose that we are using some sort of dynamically scalable data
structure, such as a hash table that we double in size every time
fullness crosses some threshold. In order to do that, we need to know
how large the table is. We can store that information, or we could just
query the size. (In this example, performance constraints might dictate
that we actually store the size of the table, but there are certainly
similar examples that wouldn't be so concerned with performance.)

I think this devolves into the poor-mans realloc. :) Instead, the code
should just realloc() when necessary and rely on the system malloc to
know when it is safe for the realloc() to be done in place.

3) This is what I want malloc_usable_size() for: garbage collection. In
order for a garbage collector to know when it should run, there needs to
be some way of tracking how much memory is in use. By using
dlsym(RTLD_NEXT, ...), I can intercept all malloc/calloc/realloc/free
calls and track current memory usage. However, unless there is a way of
getting the size of each allocation, I don't know how much to subtract
from the count for realloc/free calls.

This is a good reason, and is what I hoped you had in mind when you
proposed it. :) I think that other than this type of code though (malloc
reimplementations or wrappers), nothing else should use this function. :)

4) Porting code from Linux. Case in point: Xara Xtreme, currently being
ported by Vasil Dimov. At the moment, he has to use dlmalloc.

Does it contain a G/C of its own or some such?

Jason

Following is what I've written for the malloc(3) man page:
----
The malloc_usable_size() function returns the usable size of the
allocation pointed to by ptr. The return value may be larger than the
size that was requested during allocation. malloc_usable_size() is not
intended as a mechanism for in-place realloc(), though it can be abused
that way; rather it is primarily provided as a tool for introspection
purposes. Any discrepancy between the requested allocation size and the
size reported by malloc_usable_size() should not be depended on, since
such behavior is entirely implementation-dependent.
----

I would word it stronger: "malloc_usable_size() should not be used as a
mechanism for in-place realloc(). It is provided solely as a tool for
introspection purposes."

:)

--
John Baldwin <jhb@xxxxxxxxxxx> <>< http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve" = http://www.FreeBSD.org
_______________________________________________
freebsd-arch@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: (part 10) More Schildt-like errors in Dicky Heathens book
    ... void* q = realloc; ... So now this handles realloc(p, ... void* malloc(size_t s) ... a NULL result from malloc always indicates allocation failure, ...
    (comp.lang.c)
  • Re: resizing an array, is there a better way?
    ... >> a large virtual memory space such that they can be trivially extended. ... > when realloc() is called with a size of 13; ... > allocation is done just after the existing block, ...
    (comp.lang.c)
  • Re: Proposed addition of malloc_size_np()
    ... Since malloc_sizeis defined on at least one platform to return the requested size, maybe a name like malloc_allocated_sizewould help avoid that confusion, and make it clear that the consumer is getting back a commitment and not a hint for future realloc(), etc. ... This is what I want malloc_usable_sizefor: garbage collection. ... By using dlsym, I can intercept all malloc/calloc/realloc/free calls and track current memory usage. ... However, unless there is a way of getting the size of each allocation, I don't know how much to subtract from the count for realloc/free calls. ...
    (freebsd-arch)
  • Re: realloc but not copy
    ... Well, if a realloc fails, then a malloc then copy, then free should ... will copy the entire buffer on realloc if a data move is required. ... There is a possible approach here that will work on some heap designs. ... You can first reallocate the allocation to the *smallest* possible ...
    (comp.lang.c)
  • Re: Proposed addition of malloc_size_np()
    ... Since malloc_sizeis defined on at least one platform to return the requested size, maybe a name like malloc_allocated_sizewould help avoid that confusion, and make it clear that the consumer is getting back a commitment and not a hint for future realloc(), etc. ... Suppose that we are about to write to a string that we know is malloced, and we want to be sure that we aren't overflowing the allocated buffer. ... We can store that information, or we could just query the size. ... However, unless there is a way of getting the size of each allocation, I don't know how much to subtract from the count for realloc/free calls. ...
    (freebsd-arch)