Re: Proposed addition of malloc_size_np()

On Mar 27, 2006, at 10:50 AM, John-Mark Gurney wrote:
Jason Evans wrote this message on Mon, Mar 27, 2006 at 08:34 -0800:

I can think of a few straightforward uses:

[...]

2) [...]

This use case is incorrect...

Fair enough. That leaves three use cases.

4) Porting code from Linux. Case in point: Xara Xtreme, currently being
ported by Vasil Dimov. At the moment, he has to use dlmalloc.

sad to say, but we seem to always inherit Linux's warts in the name of
portability.. :( I'd say adopt a method similar to the ports tree, once
we have 10+ ports depending upon it, then we can consider this.. but
importing this function for a single program seems a bit excessive..

malloc_usable_size() is *not* a Linux wart. Instead, it mostly[1] addresses a fundamental flaw in the malloc(3) API. We must go back much farther than Linux if we really want to start disparaging malloc (3). Consider that the allocator must know how large allocations are so that it can do the right thing for realloc() and free(). Consider also that the application must know how large its objects are so that it can stay in bounds. That means that such information has to be stored in two places. Following is a much more reasonable API that doesn't burden the allocator with information that the application should be able to provide:

void *
malloc(size_t size);

void *
calloc(size_t size);

void *
realloc(void *ptr, size_t size, size_t old_size);

void
free(void *ptr, size_t size);

That's not what we have though, and we're certainly not going to completely redesign malloc(3) at this point. So, instead, we're faced with trying to patch things up as well as is feasible without breaking compatibility. Let's re-visit Poul-Henning's suggestion, which involves a parallel API that exposes the internal sizes to the application. The following is not identical to what he proposed, but it has the same semantics:

void *
malloc_np(size_t *size);

void *
calloc_np(size_t *size);

void *
realloc_np(void *ptr, size_t *size, size_t *old_size);

void *
memalign_np(size_t alignment, size_t *size);

void *
free_np(void *ptr, size_t *size);

This serves much the same purpose as the idealized malloc(3) API I outlined above but with the advantage that it can co-exist with the current malloc(3) API. From an engineering perspective, this is probably actually a better solution than malloc_usable_size(), but 1) it's a much broader API, 2) it's nothing like what any other OSes have done, and 3) in practice it really doesn't gain us anything over malloc_usable_size() (in fact it doesn't solve the problem of porting Linux and OS X software).

For my own personal needs, I really don't care whether we add malloc_usable_size() or the broader API described above. However, I do feel like the latter is an over-engineered solution to the problem. To me, this isn't about reinventing a perfect wheel -- it's about filing off a rough edge on the stone wheel we're stuck with.

Jason

[1] In a slightly more ideal world, we would add malloc_size() instead of malloc_usable_size(), which would return the size that was passed to malloc()/calloc()/realloc(). However, that would impose implementation constraints on the allocator that would have far- reaching implications. I don't want to get into the details here, but suffice it to say that requiring support for malloc_size() would require a lot of extra space overhead for many allocator designs, including for phkmalloc and jemalloc.

_______________________________________________
freebsd-arch@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • [md-accel PATCH 04/19] async_tx: add the async_tx api
    ... The async_tx api provides methods for describing a chain of asynchronous ... channel pool is organized as a per-cpu array of channel pointers. ... * under the terms and conditions of the GNU General Public License, ... +static void ...
    (Linux-Kernel)
  • [PATCH 11/14/] Doc. sources: expose hpet
    ... the rtc.c driver. ... There is a user space API and a kernel space API. ... -static void ...
    (Linux-Kernel)
  • Re: Help with Date and SimpleDateFormat
    ... > void applyLocalizedPattern ... Try browsing through the API docs for some classes that have only ... In a case where an API library method throws an unexplainable exception, ...
    (comp.lang.java.programmer)
  • Re: Receiving a boolean from an InputBuffer ?
    ... Of course memcpy et al are functions defined in an Application Programmers ... like any well behaved API ... void main ... >>> go and learn api programming first, but this particular api is the ...
    (microsoft.public.win32.programmer.kernel)
  • Re: [patch] mm: Reimplementation of dynamic percpu memory allocator
    ... > allocator doesn't depend on slab, so that it can be used for slab's internal ... the same per_cpu_offset be used as for the static per-cpu variables. ... +static int split_block(unsigned int i, unsigned short size, ... +static void *alloc_from_block(unsigned long size, unsigned long align, ...
    (Linux-Kernel)