Re: ULE vs. 4BSD scheduler benchmarks



On Sat, Jan 28, 2012 at 5:40 PM, Florian Smeets <flo@xxxxxxxxxxx> wrote:

[current@ bcc'ed to get a wider audience, please discuss on performance@]

Hi,

in recent times i saw a lot of threads where it was suggested people
should switch from the ULE to the 4BSD scheduler. That got me thinking
and i decided to run a few benchmarks. I looked through all the stuff
Kris and Jeff did a few years ago and tried to follow their example. The
main motivation is however that we (Attilio Rao and I) want to set a
baseline for future reference, mainly for the work that's going on in
the vmcontention branch right now, that is the reason why all tests were
run on head@r229659. All debugging was disabled (WITNESS and friends for
the kernel and MALLOC_PRODUCTION=yes for libc).

For now i ran 3 different things. MySQL/sysbench, PostgreSQL/pgbench and
pbzip2.

All software was installed from ports with the default system gcc (gcc
version 4.2.1 20070831 patched [FreeBSD]), with the exception of
PostgreSQL. I created new postgres92-{server,client} ports with a
snapshot of PostgreSQL 9.2dev from 16.01.2012, as a lot of scalability
work was done in PostgreSQL 9.2.

MySQL version 5.5.20
sysbench version 0.4.12
PostgreSQL/pgbench version 9.2dev
PBZIP2 version v1.1.6

The machine these test were run on is a 2x4 core Xeon L5310 @ 1.60GHz
with 4GB RAM. Here is the complete topology:

kern.sched.topology_spec: <groups>
<group level="1" cache-level="0">
<cpu count="8" mask="ff">0, 1, 2, 3, 4, 5, 6, 7</cpu>
<children>
<group level="2" cache-level="2">
<cpu count="4" mask="f">0, 1, 2, 3</cpu>
</group>
<group level="2" cache-level="2">
<cpu count="4" mask="f0">4, 5, 6, 7</cpu>
</group>
</children>
</group>
</groups>

The database benchmarks were all run with a work set that fit into the
configured database memory, so after the warmup phase no disk io was
involved. sysbench was run with 1 million rows, innodb was the engine we
used as Kris work already showed that it scales much better than myisam
(also innodb is the default in MySQL's 5.5 branch). Pgbench was run
using a scaling factor of 100. The connection to the databases was using
a unix socket, also only read only tests were run.

The input and output files for the pbzip2 test were on tmpfs.

The results are available in this Google docs spreadsheet, if you scroll
down there are also some nice graphs.


https://docs.google.com/spreadsheet/ccc?key=0Ai0N1xDe3uNAdDRxcVFiYjNMSnJWOTZhUWVWWlBlemc

Over time i will add more benchmarks to the doc (i.e nginx/php-fpm and
so on). I tried to run some nginx benchmarks, but those are limited by
netisr, as i did not find a web server benchmark tool which can use unix
sockets, any suggestions welcome.

The conclusion right now seems to be that ULE is faster for database
workload, but for strongly CPU-bound workloads 4BSD can be a better
choice. I can provide KTR traces and/or schedgraph output for cases
where 4BSD is better than ULE.

I want to thank Sean Bruno and Yahoo for setting up / providing the
machines to run these test on, and Attilio for suggestions and his
general helpfulness.

Florian






Please forgive me for the following message :

The work reported is so useful that I wish it should be converted to a
testing facility
applicable by the users with their own work loads and generate data to be
analyzed
statistically .

To explain what I want to say , the following part is written ...


In some messages , the people are complaining about quality of FreeBSD .

Sometimes , I also expressed a wish about a testing facility usable by the
users/installers
of the FreeBSD :

(1) There is no any hardware database maintained by the installed FreeBSD :
In some distributions , at the end of a successful installation ,
a profile of the hardfware is sent to their hardware compatibility
data base .

(2) There is NO any Handbook about testing the FreeBSD as a whole or its
parts :
When it is installed we do not know which parts will work and will not
work .
As an example , Microsoft is continuously supplying test programs to
check whether
its new version can be installed on the user computer or not without
installing it .
Such a facility does not exist for FreeBSD ( to my knowledge ) .

My wish is to have programs / scripts to be applied in such a way
that if they find
a trouble point , they will automatically generate a structured
message
( for example , in XML ) to a central FreeBSD data base .


The existing parts for testing in SVN are the following :

http://svnweb.freebsd.org/base/head/tools/test/
http://svnweb.freebsd.org/base/head/tools/regression/


There is no any guide / scripts to be applied after installation to check
the
installed system by the users ( to my knowledge ).



In Linux world , there are some sites about testing :

http://www.linuxtesting.org/
http://sourceforge.net/projects/posixtest/
http://sourceforge.net/projects/ltp/
http://sourceforge.net/projects/nptltracetool/
http://sourceforge.net/projects/tslogparser/
http://lttng.org/



As an example , the following pages may be useful :

http://www.mageia.org/en/
http://www.mageia.org/en/contribute/
https://wiki.mageia.org/en/QA_Team
https://wiki.mageia.org/en/QA_Team_portal


https://wiki.mageia.org/en/Mageia_2_Alpha_2_testing_wanted
https://wiki.mageia.org/en/Working_on_pre-release_ISOs
https://wiki.mageia.org/en/QA_process_for_testing_upgrades
https://wiki.mageia.org/en/QA_process_for_testing_installations
https://wiki.mageia.org/en/QA_process_for_validating_updates
https://wiki.mageia.org/en/QA_testing_procedures
https://wiki.mageia.org/en/Category:QA_procedures


https://wiki.mageia.org/en/Bugzilla
https://wiki.mageia.org/en/How_to_report_a_bug_properly


A similar site / pages may be generated for the FreeBSD .

There was a site for OpenSolaris as

http://test.opensolaris.org/

but now it is shut down .

The following pages are still available :

http://hub.opensolaris.org/bin/view/Community+Group+testing/downloads
http://dlc.sun.com/osol/test/downloads/current/


For the FreeBSD , there is chicken-egg problem :

Due to difficulties of use of FreeBSD , its default settings for desk top
users , difficulties about PC-BSD as an alternative desk top , in my
opinion , is
preventing wide adoption of FreeBSD .


Over the years , I am waiting that one day , I will be able to use FreeBSD
very fluently without living a night mare of setting parameters for each
installation one by one .


For example , in FreeBSD 9.0 amd64 Release :


GNOME is working with a very slow start in root login ,
completely unusable in another user login .

KDE4 , physically unusable due to waiting to start .

Fluxbox is working very well , but it is not for starters .


These problems can not be solved by the probable users .

In GNOME or KDE pages testing is requested .
I do NOT know how to test them : There is no explicit guides / scripts /
algorithms
to be applied , and reports to be generated .


I appreciate works performed by the FreeBSD developers and maintainers with
my
heartiest thanks . Now , my wish is that , in some way , the user interface
of FreeBSD
should be improved in a way that , the people will be able to install it
and
start immediately to work with it .


If I can do it myself , I could do it immediately . I am not an expert
about
operating systems . If I were , I could do the following as a starting
point :


(1) Monitors and video cards are cheap .
Attach to a computer extranous three monitors .
In Loader.conf , define these monitors for
(i) stdin
(ii) stdout
(iii) stderr
to eliminate necessity of serial console for desk top systems .

I studied SVN sources through

http://fxr.watson.org/

but , I could not find places to modify to divert the outputs to
distinct monitors .


(2) When X is starting , it is blanking the screen , and is continuing to
load the required
window manager and its parts .
During that time , the parts are generating a large number of messages
at the back ground which they are not visible .

There is a necessaity to open a messages window and display messages in
that window
to track what is going on .

(3) When X is starting , the other parts should divert their messages to
log files instead
of displaying them on the invisible back ground of X .


(4) For X , use two monitors : One is primary , and other is for messages .

If the above modifications are done , it will be possible to understand the
problems and
report them correctly .

In summary :

(1) Prepare a FreeBSD Testing Handbook .
(2) By making the above modifications , eliminate necessity of serial
console
as much as possible for desk top users .
(3) Supply scripts / algorithms , and suitable programs for testing and
automatic reporting .
(4) Apply outcomes of testing to FreeBSD .
Improve its easiness of usability .







Thnk you very much .

Mehmet Erol Sanliturk
_______________________________________________
freebsd-performance@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@xxxxxxxxxxx"



Relevant Pages

  • Running FreeBSD on IBM Blade (works)
    ... Normally FreeBSD should boot into sysinstall and keyboard should work. ... Installation from CD did not work, due to input/outpu errors of /dev/cd0. ... pci0: <PCI bus> on pcib0 ... 200.000MB/s transfers, Tagged Queueing Enabled ...
    (freebsd-stable)
  • Re: Frustration
    ... FreeBSD to version 6.1. ... be it the software installation or hardware ... I've experienced all the problems with Windows ... I've had a million problems with BSD, and with Linux, ...
    (freebsd-questions)
  • Re: New FreeBSD package system (a.k.a. Daemon Package System (dps))
    ... Recently I stumbled across a document you wrote in 2001, entitled "FreeBSD installation and package tools, past, present and future". ... I find FreeBSD appealing and I would like to contribute it its success, and as your article describes, the installation and packaging system is lacking. ... this be replaced by a single SQLite database. ...
    (freebsd-hackers)
  • Re: Drivers for my PC
    ... I installed FreeBSD 9.2 ... For example the Wi-Fi adapter, ... I converted the iso to a usb installation medium to install FreeBSD. ... You will get more support from questions@ that I added to CC, ...
    (freebsd-questions)
  • Re: My disk I/O testing methods for FreeBSD 5.3 ...
    ... Are there any outstanding code changes that will help improve ... > the performance and questions mailing lists under the title " FreeBSD ... > Operating Systems tested: ... > the default installation options as possible with no special tuning. ...
    (freebsd-performance)