Re: Massive performance loss from OS::sleep hack

On Saturday 15 September 2007 01:50:13 pm Kris Kennaway wrote:
Hi,

I have been running the volano java benchmark
(http://www.volano.com/benchmarks.html) on an 8-core i386 system, and
out of the box jdk15 on FreeBSD performs extremely poorly. The system
is more than 90% idle, and profiling shows that the ~800 threads in the
benchmark are spending most of their time doing short nanosleep() calls.


I traced it to the following FreeBSD-specific hack in the jdk:

// XXXBSD: understand meaning and workaround related to yield
...
// XXXBSD: done differently in 1.3.1, take a look
int os::sleep(Thread* thread, jlong millis, bool interruptible) {
assert(thread == Thread::current(), "thread consistency check");
...

if (millis <= 0) {
// NOTE: workaround for bug 4338139
if (thread->is_Java_thread()) {
ThreadBlockInVM tbivm((JavaThread*) thread);
// BSDXXX: Only use pthread_yield here and below if the system thread
// scheduler gives time slices to lower priority threads when yielding.
#ifdef __FreeBSD__
os_sleep(MinSleepInterval, interruptible);
#else
pthread_yield();
#endif

When I removed this hack (i.e. revert to pthread_yield()) I got an
immediate 7-fold performance increase, which brings FreeBSD performance
on par with Solaris.

What is the reason why this code is necessary? Does FreeBSD's
sched_yield() really have different semantics to the other operating
systems, or was this a libkse bug that was being worked around?

Hello Kris,

I recall why I added the os_sleep() call. While working on the certification
testing one of the JCK tests was deadlocking. The test itself was faulty in
that it created a high priority thread in a tight yield loop. Since
pthread_yield() on a single processor system will not yield to lower
priority threads, the higher priority thread effectively blocked the
lower priority thread from running and the test deadlocked.

I filed a formal appeal to have the JCK test corrected. However since the
api states:

http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Thread.html#yield()
"Causes the currently executing thread object to temporarily pause
and allow other threads to execute."

the appeal was denied. I further argued that many publications written
by or or-authored by Sun employees state that Thread.yield() can not
be relied upon in this fashion:

"However, I believe there are other less authoritative sources
of information that support the position that yield() can not
expected to allow lower priority threads to execute in all cases.
For example the following Sun document describes yield() as
advisory only and not to depend on it for correctness:

http://java.sun.com/j2se/1.5.0/docs/guide/vm/thread-priorities.html#general

The document also refers to a Sun published book, "Effective
Java Programming Language Guide" that describes yield()
should not be relied on for correctness and portability.

Another book I have specifically addresses the topic. It states
that most schedulers do not stop the yielding thread from running
in favor of a thread of lower priority. This is from the Sybex
"Complete Java 2 Certification Study Guide" co-authored by
a Sun employee.

The publications I have referred to above present a case that the
api description of yield() doesn't fully describe the expected behavior
of the function when combined with thread priorities. While they are
not the authoritive publications on the function, I think they represent
the general historical interpretation of the expected behavior."

In the end I was not able to convince Sun to change the JCK test
so the os_sleep() call was added.

I see in my notes that kse on a single cpu system had the issue
but thr didn't (this is on 6.1-release). Perhaps now that thr is the
default on 7.0 this hack can be made conditional and only applied
to < 7.0.

The following are programs I wrote when I isolated the problem.
If the c program runs ok on 7.0 for both single cpu and mp then
remove the os_sleep() and try the java program. If that works too
then you're clear to make the os_sleep() hack only apply to <
7.0 and still be able to pass the certification tests.

Regards,
-Kurt

-----------
#include <pthread.h>
#include <unistd.h>
#include <signal.h>
#include <inttypes.h>
#include <time.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

volatile int init=0;
volatile int interrupt=0;

static void *
yielder(void *arg)
{
init = 1;
while (1) {
pthread_yield();
}
}

static void
sighandler(int sig)
{
interrupt = 1;
printf("sighandler\n");
static void
sighandler(int sig)
{
interrupt = 1;
printf("sighandler\n");
}

static void
waitForInit() {
struct timespec t, rt;

static void
waitForInit() {
struct timespec t, rt;

while (init == 0) {
t.tv_sec = 0;
t.tv_nsec = 100000;
nanosleep(&t, &rt);
}
}

static void
waitForInterrupt() {
struct timespec t, rt;

while (interrupt == 0) {
t.tv_sec = 0;
t.tv_nsec = 100000;
nanosleep(&t, &rt);
}
}

int
main(int argc, char *argv[])
{
pthread_t yldr;
pthread_attr_t attr;
struct sigaction act;

/* Install a signal handler for SIGUSR1 */
sigemptyset (&act.sa_mask);
sigaddset (&act.sa_mask, SIGUSR1);
act.sa_handler = sighandler;
act.sa_flags = 0;
sigaction (SIGUSR1, &act, NULL);

pthread_attr_init(&attr);
pthread_attr_setschedpolicy(&attr, SCHED_FIFO);

pthread_create(&yldr, &attr, yielder, NULL);
pthread_setprio(yldr, 16);
waitForInit();
if(pthread_kill(yldr, SIGUSR1) != 0)
printf("pthread_kill failed with errno = %d\n", errno);
waitForInterrupt();
}
---------------------
public class yieldTest {

public static void main(String[] args) {
yielderThread thread = new yielderThread();
thread.setPriority(Thread.NORM_PRIORITY+1);
thread.start();
thread.waitForInit();
thread.interrupt();
try {
thread.join();
} catch (java.lang.InterruptedException ie) {
}
}
}

class yielderThread extends Thread {
volatile boolean init = false;

void waitForInit() {
while (!init) {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
}
}
}

public void run() {
init = true;
while(!isInterrupted()) {
yield();
}
}
}
_______________________________________________
freebsd-performance@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: JIT Optimizations Lacking!
    ... the Java VM outperformed the CLR? ... You were right -- the simple act of attaching a debugger to CLR code forces ... static void Main ... int i = 123456789; ...
    (microsoft.public.dotnet.framework.clr)
  • Re: Enhancement request
    ... Tom Anderson wrote: ... static void main(Stringargs) ... So it is more or less a random decision that main in Java need ...
    (comp.lang.java.programmer)
  • Re: For performance, write it in C - Part 2, comparing C, Ruby and Java
    ... I didn't post the Java version because, code wise, it is pretty much a line for line translation of the C version. ... private static int NumberOfPermutations = 6; ... public static void main{ ...
    (comp.lang.ruby)
  • Re: Pls Fix my java code (using Linded List)
    ... I have a problem on my java program. ... public static void main throws IOException { ...
    (comp.lang.java.programmer)