Re: ULE vs. 4BSD scheduler benchmarks

On Sat, Jan 28, 2012 at 5:40 PM, Florian Smeets <flo@xxxxxxxxxxx> wrote:

[current@ bcc'ed to get a wider audience, please discuss on performance@]


in recent times i saw a lot of threads where it was suggested people
should switch from the ULE to the 4BSD scheduler. That got me thinking
and i decided to run a few benchmarks. I looked through all the stuff
Kris and Jeff did a few years ago and tried to follow their example. The
main motivation is however that we (Attilio Rao and I) want to set a
baseline for future reference, mainly for the work that's going on in
the vmcontention branch right now, that is the reason why all tests were
run on head@r229659. All debugging was disabled (WITNESS and friends for
the kernel and MALLOC_PRODUCTION=yes for libc).

For now i ran 3 different things. MySQL/sysbench, PostgreSQL/pgbench and

All software was installed from ports with the default system gcc (gcc
version 4.2.1 20070831 patched [FreeBSD]), with the exception of
PostgreSQL. I created new postgres92-{server,client} ports with a
snapshot of PostgreSQL 9.2dev from 16.01.2012, as a lot of scalability
work was done in PostgreSQL 9.2.

MySQL version 5.5.20
sysbench version 0.4.12
PostgreSQL/pgbench version 9.2dev
PBZIP2 version v1.1.6

The machine these test were run on is a 2x4 core Xeon L5310 @ 1.60GHz
with 4GB RAM. Here is the complete topology:

kern.sched.topology_spec: <groups>
<group level="1" cache-level="0">
<cpu count="8" mask="ff">0, 1, 2, 3, 4, 5, 6, 7</cpu>
<group level="2" cache-level="2">
<cpu count="4" mask="f">0, 1, 2, 3</cpu>
<group level="2" cache-level="2">
<cpu count="4" mask="f0">4, 5, 6, 7</cpu>

The database benchmarks were all run with a work set that fit into the
configured database memory, so after the warmup phase no disk io was
involved. sysbench was run with 1 million rows, innodb was the engine we
used as Kris work already showed that it scales much better than myisam
(also innodb is the default in MySQL's 5.5 branch). Pgbench was run
using a scaling factor of 100. The connection to the databases was using
a unix socket, also only read only tests were run.

The input and output files for the pbzip2 test were on tmpfs.

The results are available in this Google docs spreadsheet, if you scroll
down there are also some nice graphs.

Over time i will add more benchmarks to the doc (i.e nginx/php-fpm and
so on). I tried to run some nginx benchmarks, but those are limited by
netisr, as i did not find a web server benchmark tool which can use unix
sockets, any suggestions welcome.

The conclusion right now seems to be that ULE is faster for database
workload, but for strongly CPU-bound workloads 4BSD can be a better
choice. I can provide KTR traces and/or schedgraph output for cases
where 4BSD is better than ULE.

I want to thank Sean Bruno and Yahoo for setting up / providing the
machines to run these test on, and Attilio for suggestions and his
general helpfulness.


Please forgive me for the following message :

The work reported is so useful that I wish it should be converted to a
testing facility
applicable by the users with their own work loads and generate data to be
statistically .

To explain what I want to say , the following part is written ...

In some messages , the people are complaining about quality of FreeBSD .

Sometimes , I also expressed a wish about a testing facility usable by the
of the FreeBSD :

(1) There is no any hardware database maintained by the installed FreeBSD :
In some distributions , at the end of a successful installation ,
a profile of the hardfware is sent to their hardware compatibility
data base .

(2) There is NO any Handbook about testing the FreeBSD as a whole or its
parts :
When it is installed we do not know which parts will work and will not
work .
As an example , Microsoft is continuously supplying test programs to
check whether
its new version can be installed on the user computer or not without
installing it .
Such a facility does not exist for FreeBSD ( to my knowledge ) .

My wish is to have programs / scripts to be applied in such a way
that if they find
a trouble point , they will automatically generate a structured
( for example , in XML ) to a central FreeBSD data base .

The existing parts for testing in SVN are the following :

There is no any guide / scripts to be applied after installation to check
installed system by the users ( to my knowledge ).

In Linux world , there are some sites about testing :

As an example , the following pages may be useful :

A similar site / pages may be generated for the FreeBSD .

There was a site for OpenSolaris as

but now it is shut down .

The following pages are still available :

For the FreeBSD , there is chicken-egg problem :

Due to difficulties of use of FreeBSD , its default settings for desk top
users , difficulties about PC-BSD as an alternative desk top , in my
opinion , is
preventing wide adoption of FreeBSD .

Over the years , I am waiting that one day , I will be able to use FreeBSD
very fluently without living a night mare of setting parameters for each
installation one by one .

For example , in FreeBSD 9.0 amd64 Release :

GNOME is working with a very slow start in root login ,
completely unusable in another user login .

KDE4 , physically unusable due to waiting to start .

Fluxbox is working very well , but it is not for starters .

These problems can not be solved by the probable users .

In GNOME or KDE pages testing is requested .
I do NOT know how to test them : There is no explicit guides / scripts /
to be applied , and reports to be generated .

I appreciate works performed by the FreeBSD developers and maintainers with
heartiest thanks . Now , my wish is that , in some way , the user interface
of FreeBSD
should be improved in a way that , the people will be able to install it
start immediately to work with it .

If I can do it myself , I could do it immediately . I am not an expert
operating systems . If I were , I could do the following as a starting
point :

(1) Monitors and video cards are cheap .
Attach to a computer extranous three monitors .
In Loader.conf , define these monitors for
(i) stdin
(ii) stdout
(iii) stderr
to eliminate necessity of serial console for desk top systems .

I studied SVN sources through

but , I could not find places to modify to divert the outputs to
distinct monitors .

(2) When X is starting , it is blanking the screen , and is continuing to
load the required
window manager and its parts .
During that time , the parts are generating a large number of messages
at the back ground which they are not visible .

There is a necessaity to open a messages window and display messages in
that window
to track what is going on .

(3) When X is starting , the other parts should divert their messages to
log files instead
of displaying them on the invisible back ground of X .

(4) For X , use two monitors : One is primary , and other is for messages .

If the above modifications are done , it will be possible to understand the
problems and
report them correctly .

In summary :

(1) Prepare a FreeBSD Testing Handbook .
(2) By making the above modifications , eliminate necessity of serial
as much as possible for desk top users .
(3) Supply scripts / algorithms , and suitable programs for testing and
automatic reporting .
(4) Apply outcomes of testing to FreeBSD .
Improve its easiness of usability .

Thnk you very much .

Mehmet Erol Sanliturk
freebsd-performance@xxxxxxxxxxx mailing list
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@xxxxxxxxxxx"