Re: More ULE bugs fixed.

From: David Xu (davidxu_at_viatech.com.cn)
Date: 10/16/03

  • Next message: Scott: "ATA Hard Lock during boot"
    Date: Thu, 16 Oct 2003 09:49:57 +0800
    To: Julian Elischer <julian@elischer.org>
    
    

    Julian Elischer wrote:

    >On Wed, 15 Oct 2003, Daniel Eischen wrote:
    >
    >
    >
    >>On Wed, 15 Oct 2003, Jeff Roberson wrote:
    >>
    >>
    >>
    >>>I fixed two bugs that were exposed due to more of the kernel running
    >>>outside of Giant. ULE had some issues with priority propagation that
    >>>stopped it from working very well.
    >>>
    >>>Things should be much improved. Feedback, as always, is welcome. I'd
    >>>like to look into making this the default scheduler for 5.2 if things
    >>>start looking up. I hope that scares you all into using it more. :-)
    >>>
    >>>
    >>Before you do that, can you look into changing the scheduler
    >>interfaces to address David Xu's concern with it being
    >>suboptimal for KSE processes?
    >>
    >>
    >>
    >There is also some work that I'd like to get done re:
    >cleaning up the scheduler interface a bit..
    >
    >I know that Jeff and I have doiscussed this before but it was a long
    >time ago, and I've forgotten a lot and also learned a bit since then..
    >
    >Here's my logic on the matter:
    >
    >Any process has a number (fixed or variable) of kernel entities tghat
    >can be scheduled. In KSE (gotta get a better name) there are a variable
    >number of them. In libthr they are 1:1.
    >
    >I would postulate that the action of scheduling these items in a fair
    >way is up to the scheduler. I had a very crude fairness module
    >added to the BSD4.4 scheduler but I think that fairness
    >is a property of the scheduler and not of the threading package.
    >
    >If the scheduler doesn't care if threads are scheduled fairly than it
    >can just schedule all threads equally. I would say that the ksegrp
    >in question (which represents a rough unit of 'fairness'), should
    >make a call to the scheduler on creation specifying the required
    >concurrancy.
    >
    >At the moment KSE-M:N based ksegrps would specify N = NCPU, and
    >THR based ksegrps would specify N = NTHREADS.
    >KSE-1:1 runs with a KSEGRP with a concurrancy of 1 per thread.
    >
    >(I still think that THR should allocate a KSEGRP per thread not a KSE
    >but it's not critical.)
    >
    >Basically What I'm saying is that each scheduler should taka
    >a concurrency setting for each KSEGRP and how it implements it
    >is hidden from higher layers. The current 4.4 scheduler would
    >implement it using KSEs and the existitng code but other shcedulers may
    >chose to implement it in different manners.
    >
    >I think the top layer API calls for the scheduler should be:
    >setrunnable(thread)
    >choosethread()
    >sched_clocktick()
    >sched_set_concurrancy()
    >(plus all the other 'entrypoints')
    >
    >
    >I think that the scheduler needs to be in control of scheduling threads
    >because there is too much inside information needed for it to be done
    >properly by an outside entity. For example if the scheduler is not a
    >priority based scheduler then an outside entity can not know
    >how to juggle which thread should be run next if there is a choice of
    >which to do..
    >
    >this would mean that each scheduler would neeed its own module to
    >do this juggling instead of having a separate module to do it..
    >
    >it makes the job of the scheduler more difficult, but in fact it has to
    >be so, because true posix process-scope threads require that the
    >scheduler do this work.
    >
    >
    >a thread is made runnable (with a unix priority)
    >the scheduler needs to look at this thread in the context of all the
    >other threads from this process, the current concurrency rule for that
    >ksegrp and the other runnable threads, and adjust things so that:
    >1/ the new thread is run some time
    >2/ the ksegrp doesn't get TOO MUCH cpu, possibly
    >punishing other threads in the group to compensate..
    >
    >This is all up for discussion, but it's my current thinking.
    >
    >
    These are what I want to see. Current I am forced to maintain nexus
    among kse and ksegrp and
    thread whenever kse_create is called or thread is exiting but those
    nexuses is not used by
    SA code at all. From my view as a scheduler interface user, I only
    want to see
    ksegrp and thread, the scheduler interface should be thread friendly
    not kse which is
    scheduler internal detail to maintian scheduling faireness, for
    example, it might just
    be a token to assign a thread, a thread has this token can be picked
    to run on physical CPU,
    scheduler would maintain token assigning faireness, when there more
    tokens in ksegrp,
    the more concurrent level the ksegrp will get.

    Another problem is libkse in future would support cpu affinity, with
    current scheduler interface,
    I can not specify a newly create thread to be bound to a specified CPU,
    this is needed by libkse,
    when an upcall is scheduled, I want to specify which cpu is prefered to
    run newly created upcall
    thread to give userland scheduler a stable cpu affinity state.
    Dan ever asked to add kse_bind syscall to let him bind userland kse to a
    specified cpu,
    if we want to support him, then we might need this feature in kernel.

    >Julian
    >
    >
    David Xu

    _______________________________________________
    freebsd-current@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-current
    To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"


  • Next message: Scott: "ATA Hard Lock during boot"

    Relevant Pages

    • Comments on the KSE option
      ... John, I appreciate that you have made KSE an option, but the way you have done it shows a complete misundertanding of what is there. ... A naive scheduler would give you 1000 cpu slots and me 1. ... The fairness algorythm that you have made 'optional' is a very crude one and I had thought that by now someone would have written a better one, ... work only to discover at the end that it is the only thread on the ksegrp, and therefore always eligible to run). ...
      (freebsd-current)
    • Re: Code for review.. threads vs. the scheduler
      ... rather than linked with them) even knows what a struct kse looks like. ... these patches were not widly distributed. ... At this point each scheduler can be alterred almost completely ... ksegrp structures that reference kses into ...
      (freebsd-current)
    • Re: whats the different between sched_core.c and sched_ule.c?
      ... maintains a set of KSE together, ... For Linux kernel 2.6 there were two competing ... Now, for FreeBSD, several benchmarks about threaded programs have circulated, ... And this is *not* the case of the BSD scheduler. ...
      (comp.unix.bsd.freebsd.misc)
    • Re: Code for review.. threads vs. the scheduler
      ... > removed from proc.h and made a private structure within each ... > rather than linked with them) even knows what a struct kse looks like. ... > At this point each scheduler can be alterred almost completely ... > ksegrp structures that reference kses into ...
      (freebsd-current)
    • Re: More ULE bugs fixed.
      ... cleaning up the scheduler interface a bit.. ... In KSE there are a variable ... KSE-1:1 runs with a KSEGRP with a concurrancy of 1 per thread. ...
      (freebsd-current)