Re: Comments on the KSE option

Alexandre "Sunny" Kovalenko wrote:
On Fri, 2006-10-27 at 20:51 -0700, Julian Elischer wrote:
Alexandre "Sunny" Kovalenko wrote:
On Fri, 2006-10-27 at 18:25 -0700, Julian Elischer wrote:
Alexandre "Sunny" Kovalenko wrote:
On Fri, 2006-10-27 at 16:41 -0400, Daniel Eischen wrote:
On Fri, 27 Oct 2006, Paul Allen wrote:

From Julian Elischer <julian@xxxxxxxxxxxx>, Fri, Oct 27, 2006 at 12:27:14PM -0700:
The aim of the fair scheduling code is to ensure that if you, as a user,
make a process that starts 1000 threads, and I as a user, make an
unthreaded process, then I can still get to the CPU at somewhat similar
rates to you. A naive scheduler would give you 1000 cpu slots and me 1.
Ah. Let me be one of the first to take a crack at attacking this idea as
a mistake.
No, it is POSIX. You, the application, can write a program with
system scope or process scope threads and get whatever you behavior
you want, within rlimits of course.

If you want unfair scheduling, then create your threads with
system scope contention, otherwise use process scope. The
kernel should be designed to allow both, and have adjustable
limits in place for (at least) system scope threads.

Noone is saying that you can't have as many system scope threads
as you want (and as allowed by limits), just that you must also
be able to have process scope threads (with probably higher limits
or possibly no limits).

I might be missing something here, but OP was separating M:N (which is
what you are referring to above), and "fairness" (not giving process
with 1000 *system scope* threads 1000 CPU scheduling slots). As far as I
know the first one is POSIX and the second one is not.

FWIW: as an application programmer who spent considerable amount of time
lately trying to make heavily multithreaded application run most
efficiently on 32-way machine, I would rather not have to deal with
"fairness" -- M:N is bad enough.

no, fairness is making sure that 1000 process scope threads
do not negatively impact other processes.
1000 system scope threads are controlled by your ulimit settings
(Each one counts as a process.)


I apologize for misinterpreting your words. But then, if I have M:N set
to 10:1, I would expect application with 1000 process scope threads to
have as many CPU slots as 100 processes, or, if I have 10 system scope
threads and 990 process scope threads, I would expect application to
have as many CPU slots as 109 processes. Is this what you refer to as
"fairness"?

M:N is not a ratio, but rather the notation to say that M user threads are enacted using N kernel schedulable entities (kernel threads).
usually N is limited to something like NCPU kernel schedulable entities running at a time. (not including sleeping threads waiting for IO)
(NCPU is the number of CPUs).

so in fact M:N is usually M user threads over over some number like 4 or 8 kernel threads (depending on #cpus) plus the number of threads waiting for IO.

Julian
In the environment I am most familiar with -- IBM AIX -- M:N is the
ratio and it is settable either systemwide or for the specific user
using environment variable, e.g.:

export AIXTHREAD_MNRATIO=8:1

with the minimum kernel threads allocated according to another setting:

export AIXTHREAD_MINKTHREADS=4

Neither one depends on the physical CPU count in the box (which could
change in the middle of application execution anyway).

Both settings have known default values (8:1 and 8 respectively).

Between the two I can always tell how many kernel threads given amount
of the process scope threads will use. This gives me both flexibility
and predictability.

Am I understanding correctly that what you have implemented fixes number
of kernel threads at boot time and changes the M:N ratio throughout the
run time of the application?

no. the "number of available slots" for concurency is a number that is tunable on the fly. I think we may even be able to tune it in a program but I would have to look to see if that ever got committed.


_______________________________________________
freebsd-current@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: Comments on the KSE option
    ... The aim of the fair scheduling code is to ensure that if you, as a user, ... system scope contention, otherwise use process scope. ... limits in place for system scope threads. ... are enacted using N kernel schedulable entities. ...
    (freebsd-current)
  • Re: Comments on the KSE option
    ... The aim of the fair scheduling code is to ensure that if you, as a user, ... system scope contention, otherwise use process scope. ... limits in place for system scope threads. ...
    (freebsd-current)
  • Re: Comments on the KSE option
    ... The aim of the fair scheduling code is to ensure that if you, as a user, ... A naive scheduler would give you 1000 cpu slots and me 1. ... system scope contention, otherwise use process scope. ... limits in place for system scope threads. ...
    (freebsd-current)
  • Re: Comments on the KSE option
    ... :system scope contention, otherwise use process scope. ... :limits in place for system scope threads. ...
    (freebsd-current)