Re: KSE, libpthread & libthr: almost newbie question


On Fri, 27 Oct 2006, Lev Serebryakov wrote:

I've was sure, that both libpthread and libthr use KSE to make multithreading. They use KSE in different ways: libpthread uses N:M model and libthr uses 1:1 model, but bot use KSE to work.
How will be possible to sue these libraries (read: multithreaded programs) when KSE will be optional, on kernel without KSE?!

The answer to that rather critical question, not surprisingly, is complex. :-)

The FreeBSD kernel actually implements a good three different threading models (at least):

(1) Linux threads/rfork threads. These are based on the idea of multiple
"struct proc"s floating around with varying degrees of sharing.
Typically, they share file descriptor array, address space, etc. For
those familiar with the Linux clone() system call, rfork() is notionally
very similar. This model implies 1:1 threading, and leaves quite a bit to
be desired in terms of overhead, as well as having some odd implications
for POSIX-like compliance (i.e., the implementation of getpid()).

(2) KSE. This provides a kernel<->userspace threading framework allowing the
implementation of a broad array of threading and scheduling models. This
is a fairly complex, but very flexible model, which has several levels of
layering required to implement different scheduling policies using M:N
threading. Notice that 1:1 scheduling is a subset of M:N, so you can
implement 1:1 in this manner. One of the significant concerns about KSE
is that it adds a great deal of complexity to the kernel scheduler
architecture, making it quite difficult to optimize, further granularize
the locking for, understand, etc.

(3) thr. This is a simpler 1:1 threading API to the kernel, which make use of
the same architectural structures present in the kernel for KSE, but
without the full capability of M:N threading. In particular, it has
simplifying assumptions regarding how user threads enter the kernel and
are mapped into kernel threads, so doesn't need an upcall mechanism, or
create new threads when an existing thread sleeps.

The NO_KSE patch disables the code paths required only for (2), not for (1) or (3). One of the current theories bouncing around the kernel developer community is that the complexity and overhead of (2) outweighs many of the benefits of KSE, and that by making it an option, we can better evaluate the impact. Notice that this isn't just about code complexity, but also about scheduler overhead. David Xu has reported a non-trivial performance change from the reduced overhead of the scheduler paths. So now we're at a point where we can more fully evaluate the impact of KSE (since we can actually compile it out of the scheduler). Before anything further can be done, we now need to do that evaluation.

Robert N M Watson
Computer Laboratory
University of Cambridge
_______________________________________________
freebsd-current@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: KSE, libpthread & libthr: almost newbie question
    ... They use KSE in different ways: libpthread uses N:M model and libthr uses 1:1 model, but bot use KSE to work. ... How will be possible to sue these libraries when KSE will be optional, on kernel without KSE?! ... The FreeBSD kernel actually implements a good three different threading models: ... Notice that this isn't just about code complexity, but also about scheduler overhead. ...
    (freebsd-current)
  • Re: effect of sched_set{param|scheduler} on MT-process
    ... Couldn't we design a scheduler which knows about ... > groups of kernel threads representing a process? ... 1:1 model means that one thread is mapped to one KSE ... (Kernel Schedulable Entity). ...
    (comp.unix.programmer)
  • HEADSUP: KSE support removed from the kernel.
    ... CURRENT is no longer going to support KSE threaded applications. ... As mentioned in UPDATING the following libmap.conf should work along with compat libs for old dynamically linked libraries: ... Statically linked binaries will no longer work. ... Remove kernel support for M:N threading. ...
    (freebsd-current)
  • Re: libthread 1:1 threads
    ... >threading caught my eye. ... KSE and 1:1 threading are different things. ... One creatres kernel threads on demand and the other keeps the kernel ... rfork is not the same.. ...
    (freebsd-hackers)
  • Re: libthread 1:1 threads
    ... >>threading caught my eye. ... > KSE and 1:1 threading are different things. ... > One creatres kernel threads on demand and the other keeps the kernel ... use it to modify an extant process. ...
    (freebsd-hackers)