Re: Anyone interested in improving postgresql scaling?

On Tue, Apr 10, 2007 at 12:04:32AM -0400, Kevin Way wrote:
Kris Kennaway wrote:
If so, then your task is the following:

Make SYSV semaphores less dumb about process wakeups. Currently
whenever the semaphore state changes, all processes sleeping on the
semaphore are woken, even if we only have released enough resources
for one waiting process to claim. i.e. there is a thundering herd
wakeup situation which destroys performance at high loads. Fixing
this will involve replacing the wakeup() calls with appropriate
amounts of wakeup_one().
Could this cause problem cause a situation where an 8-Core system was
50-75% slower than an otherwise equivalent 2-Core system?

I have a graph of my sysbench/pgsql results here:

http://blog.insidesystems.net/files/sysctl-pgsq-amd64-wtf.png

As the graph shows, the 8-core system is about half the speed of the
2-core system at 2 simultaneous threads, and it decays down to
approximately 1/4 the speed of the 2-core system as the # of threads hits 5.

All other (non-pgsql, non-sysv) tests came back approximately as
expected, but I'm left wondering if I did something wrong, or if 8 cpus
are slower than 2, when it comes to Postgres on currently available FreeBSD.

I wouldn't expect 6.2 to have good scaling on 8 cpus for this kind of
benchmark. Fixing that was what we have been working on for 7.0 (and
basically succeeded). The most important fixes are already in 7.0
CVS, but a possible merge to 6.x will only happen after a suitably
conservative delay.

Kris

Attachment: pgpXMsrJoIDHK.pgp
Description: PGP signature

Relevant Pages

  • Re: Anyone interested in improving postgresql scaling?
    ... whenever the semaphore state changes, ... wakeup situation which destroys performance at high loads. ... As the graph shows, the 8-core system is about half the speed of the 2-core system at 2 simultaneous threads, and it decays down to ... All other tests came back approximately as expected, but I'm left wondering if I did something wrong, or if 8 cpus are slower than 2, when it comes to Postgres on currently available FreeBSD. ...
    (freebsd-performance)
  • Re: Anyone interested in improving postgresql scaling?
    ... Robert Watson wrote: ... Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough resources for one waiting process to claim. ... Fixing this will involve replacing the wakeup() calls with appropriate amounts of wakeup_one. ... All other tests came back approximately as expected, but I'm left wondering if I did something wrong, or if 8 cpus are slower than 2, when it comes to Postgres on currently available FreeBSD. ...
    (freebsd-performance)
  • Re: Anyone interested in improving postgresql scaling?
    ... Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough resources for one waiting process to claim. ... Fixing this will involve replacing the wakeup() calls with appropriate amounts of wakeup_one. ... All other tests came back approximately as expected, but I'm left wondering if I did something wrong, or if 8 cpus are slower than 2, when it comes to Postgres on currently available FreeBSD. ... Kevin Way Inside Systems, Inc. ...
    (freebsd-performance)
  • Re: Anyone interested in improving postgresql scaling?
    ... whenever the semaphore state changes, ... wakeup situation which destroys performance at high loads. ... active postgresql processes). ...
    (freebsd-current)
  • Re: Anyone interested in improving postgresql scaling?
    ... whenever the semaphore state changes, ... wakeup situation which destroys performance at high loads. ... active postgresql processes). ...
    (freebsd-performance)