Re: ten thousand small processes

• Previous message: Terry Lambert: "Re: The dangers of replacing malloc()"
• In reply to: D. J. Bernstein: "Re: ten thousand small processes"
• Next in thread: Jeff Roberson: "Re: ten thousand small processes"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Date: Thu, 26 Jun 2003 15:04:41 -0400
To: freebsd-performance@freebsd.org

D. J. Bernstein wrote:
>> Remember that VMM hardware requires page-alignment
>
> When I ask why the stack and data aren't put on the same page, and you
> say ``They aren't on the same page,'' you aren't answering the question.

True. If A implies B, yet A is false, what have you shown about B?

I didn't write the words "They aren't on the same page". What I wrote was
"stack and data have different VM protections and, since VMM hardware requires
page alignment, the stack and data need to be put on different pages."

If you don't think the latter paraphrase is a valid answer to the question you
asked, fine, say so: quoting a single phrase from the three paragraphs I wrote,
and then adapting it to the point where you can complain that what I didn't say
didn't answer your question has more to do with rhetorical strawmen. Thanks.

> (As for adding an x bit to data: This obviously won't break anything.)

Using VMM protection to forbid code execution within the DATA, BSS, heap, and
stack (if one can) mitigates against a common class of problems-- "buffer
overflows"-- which have lead to a vast number of security vulnerabilities.

Well-written programs do not require VMM hardware to protect them from such
issues, but that doesn't mean that a general-purpose OS like FreeBSD should
assume that all programs are well-written.

[ ... ]
>> It's easy to write a memory allocator that performs a specific case well;
>> writing a general purpose malloc is significantly more complicated,
>
> I'm not talking about replacing malloc() with a special-purpose
> allocator. I'm talking about adding a tiny bit of code to malloc() to
> magically take advantage of space that is being ignored right now.
>
> The savings in this situation go beyond those dozens of megabytes of
> magically reacquired RAM. There's a nasty spike in memory usage as soon
> as malloc() starts extending the heap; when a program's allocations fit
> into the magically reacquired RAM, the program also avoids the spike.

Calling malloc(1) for the first time causes a 16K spike in memory usage under
FreeBSD 4.8; that's a factor of four larger than the minimum possible allocation
of one 4K page.

[ It's not clear whether all four VM pages are actually allocated, or whether
one is and three are waiting for a write-fault before the system allocates
resources for them. It's also worth noting that malloc() has to return memory
properly aligned to whatever the local hardware requires for arbitrary data
types: malloc(1), malloc(4), and even malloc(16) might all reserve the same
amount of space. ]

With regard to your magic suggestion, frankly, what you are doing is solving a
much easier problem domain than the general purpose malloc is expected to be
capable of solving. Linear memory allocation and no provision to handle
internal fragmentation (because you'll never release or reuse in this buffer of
memory, yes?) are wonderful decisions for a memory allocator tuned for a very
small program. So long as the program doesn't repeatedly allocate and free this
memory, anyway.

-- 
-Chuck
_______________________________________________
freebsd-performance@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org"

• Previous message: Terry Lambert: "Re: The dangers of replacing malloc()"
• In reply to: D. J. Bernstein: "Re: ten thousand small processes"
• Next in thread: Jeff Roberson: "Re: ten thousand small processes"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]