Re: Anyone interested in improving postgresql scaling?

Kris Kennaway wrote:
On Tue, Apr 17, 2007 at 02:51:55PM -0400, Kevin Way wrote:

I built 7.0 as of 6 days ago, and ran the same test using 8-cores, ULE and 4BSD. The results are available at:

http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0

Unfortunately, I can't run the additional tests, as the machines in question have now been deployed to production. I should have similar equipment available in a few weeks, and will try the other tests at that point.

That's better but still not comparable to my results. One important
change you may have missed was to enable MALLOC_PRODUCTION in
/usr/src/lib/libc/stdlib/malloc.c. This disables malloc debugging
which may have significant overhead.

My results from about a month ago:

http://people.freebsd.org/~kris/scaling/nickel.png

The big dip between 5-7 clients is believed to be due to a ULE bug
that has known cause but is not yet fixed (on dual core CPUs processes
are bouncing around between the two cores too much). The main point
is that apart from this anomaly in the middle there is approximately
linear scaling from 1 to 8 CPUs. On your system it is not scaling
linearly beyond 4.

I do have some other changes in my kernel but I don't think any of
them are enough to explain this difference.

Anyway, get back to us when you can retest with the above change.


Confirmed, I did miss MALLOC_PRODUCTION. I incorrectly believed that I just needed an empty malloc.conf. This is somewhat exciting news from my vantage point. I'll post a corrected re-test as soon as I have another available 8-core machine.

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

Relevant Pages

  • Re: Anyone interested in improving postgresql scaling?
    ... Unfortunately, I can't run the additional tests, as the machines in ... question have now been deployed to production. ... that has known cause but is not yet fixed (on dual core CPUs processes ... linear scaling from 1 to 8 CPUs. ...
    (freebsd-performance)
  • Re: Compiling Packages
    ... Kevin D. Kinsey, DaleCo, S.P. wrote: ... > Kris Kennaway wrote: ... >> machines with good network connectivity. ...
    (freebsd-questions)
  • Re: Performance of 4.x vs 5.x (Re: Lifetime of FreeBSD branches)
    ... > Kris Kennaway wrote: ... >> they build world or a kernel and notice that it's taking much longer ... I don't run FreeBSD on my desktop machines so I haven't seen this ... One obvious guess is that it's due to VFS being under Giant, ...
    (freebsd-stable)
  • Re: 6.0 Release - Pentium install panic and some questions
    ... On Tue, 22 Nov 2005 07:03 am, Kris Kennaway wrote: ... When booting from floppy on these ... other machines it says "Boot FreeBSD with ACPI disabled". ... page fault while in kernel mode ...
    (freebsd-stable)
  • Re: Softupdates not preventing lengthy fsck
    ... On 11 Apr, Kris Kennaway wrote: ... > I'm seeing the following problem: on 6.0 machines which have had a lot ... > It's as if dirty buffers aren't being written out properly, ... This looks a lot like it could be a vnode refcnt leak. ...
    (freebsd-current)