Re: Comments on the KSE option


On Sat, 28 Oct 2006, Paul Allen wrote:

From Robert Watson <rwatson@xxxxxxxxxxx>, Sat, Oct 28, 2006 at 11:04:48AM +0100:
This is my single biggest concern: our scheduling, thread/process, and context management paths in the kernel are currently extremely complex. This has a number of impacts: it makes it extremely hard to read and understand, it adds significant overhead, and it makes it quite hard to modify and optimize for increasing numbers of processors. We need to be planning on a world of 128 hardware threads/machine on commodity server hardware in the immediate future, which means that the current "giant sched_lock" cannot continue much longer. Kip's prototypes of breaking out sched_lock as part of the sun4v work have been able to benefit significantly from the reduced complexity of a KSE-free kernel, and it's fairly clear that the task of improving schedule scalability is dramatically simpler when the kernel model for threading is more simple. Regardless of where the specific NO_KSE option in the kernel goes, reducing kernel scheduler/etc complexity should be a first order of business, because effective SMP work really depends on that happening.

Let us suppose that this M:N business is important, perhaps something to consider is why and whether the kernel has so much knowledge of it.

If I read Matt Dillon's comment closely enough, I believe his precise recommendation was not "something like kse as Julian read it" but rather something where this M:N component was entirely part of the userland threading support and therefore would just go away or not depending on which library you linked with.

I think posix might require a global priority space though...

Anyways it remains dubious in my mind that the kernel should allow a user to create many processes but penalize creating threads.

The only reason I can think of is that you expect people to be sloppy with their threads and careful with their processes.

Still if I am ray-tracing why should I need to make a point of picking my thread/process balance to get around your mechanism. If fairness is the goal why am I even allowed to do so?

I think the notion of fairness is orthogonal to M:N threading. M:N is about efficiently representing user threading to kernel space, as well as avoiding kernel involvement in user context switches when not needed. Fairness is about how the kernel allocates time slices to user processes/threads. Fairness can be implemented for both 1:1 and M:N, with the primary differences being in bookkeeping.

Programmers often use threading based on a misunderstanding about performance. Threading actually increases kernel lock contention when compared with using processes for parallelism, so if the benefits from address space sharing don't outweigh the added costs of contention, threaded applications can run significantly slower than multi-process ones. Many programmers believe that threading is necessarily faster than using multiple processors, so I think we're fundamentally forced to deal with the way they do use them, rather than how they should use them.

Robert N M Watson
Computer Laboratory
University of Cambridge
_______________________________________________
freebsd-current@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: Comments on the KSE option
    ... context management paths in the kernel are currently extremely complex. ... This has a number of impacts: it makes it extremely hard to read and ... significantly from the reduced complexity of a KSE-free kernel, ... dramatically simpler when the kernel model for threading is more simple. ...
    (freebsd-current)
  • Re: KSE, libpthread & libthr: almost newbie question
    ... They use KSE in different ways: libpthread uses N:M model and libthr uses 1:1 model, but bot use KSE to work. ... How will be possible to sue these libraries when KSE will be optional, on kernel without KSE?! ... The FreeBSD kernel actually implements a good three different threading models: ... Notice that this isn't just about code complexity, but also about scheduler overhead. ...
    (freebsd-current)
  • Re: program starts with 3 threads
    ... > application which uses the daemon, they both start with 3 processes. ... Regards, Jono. ... With normal versions of the Linux 2.4.x kernel, ... The new Linux 2.6.x kernel has a completely rewritten threading ...
    (Debian-User)
  • Re: how do threads work?
    ... So one might as well move the entire threading into ... The kernel already has most of the threading ... pusher started ... poper started ...
    (comp.os.linux.development.system)
  • Re: KSE, libpthread & libthr: almost newbie question
    ... A 1:1 threading is much simpler than N:M threading, ... and thus doesn't require KSE support in the kernel; ... Without the KSE option in the kernel, ... I can even remember libpthread being originaly named libkse. ...
    (freebsd-current)