Re: cpuset and affinity implementation

On Mon, 25 Feb 2008, Alfred Perlstein wrote:

* Jeff Roberson <jroberson@xxxxxxxxxxxxxx> [080224 02:39] wrote:
Please see:
http://people.freebsd.org/~jeff/cpuset.diff

This is unfortunately intertwined with ULE's new CPU selection algorithm
so that code is in the patch as well. Otherwise, this includes a simple,
ugly userland tool called cpuset and all of the kernel support required.
I have tested this by creating sets and subsets and modifying their cpu
masks under load. I'm able to dynamically reprovision without issue.

This doesn't have support for jails but the infrastructure is there. It
also fails to modify sets if it would leave threads without a valid cpu
to run on. I have not implemented a force option but it will be trivial
to do so. The initial cpu set is also created before we know all_cpus so
it's faked up with all cpus set for now.

I mostly want people to look at the interface in cpuset.h and make sure
they agree with it before I start polishing to commit. I'm fairly happy
with the way the syscall api looks now. The code itself ended up being
much more complicated than I'd hoped due to locking considerations. Try
not to look at cpuset_setproc() ;).


Jeff, this is very cool. I do have one issue though:

+ * A thread may not be assigned to a a group seperate from other threads in
+ * the process. This is to remove ambiguity when the setid is queried with
+ * a pid argument. There is no other technical limitation.

Am I understanding things correctly such that within a process there
can only be one "set"?

If so this restricts some of the benifits you get with sets and
binding.

An example would be some sort of system with multiple CPUs where some
are assigned specifically for pseudo-realtime processing and others are for
general control things such as cli, stats, etc.

In our case we would like to be able to run some threads on specific
cpu sets, and other threads to be run anywhere on the control CPUs.

Can this be done with this API?

Individual threads can be bound to any cpu or group of cpus within the set. So if you just make a set that includes all cpus in the system you can then bind your realtime threads to specific cpus and the other threads to the remainder. You will have to specifically bind each thread however.

The reason individual threads can't be assigned to groups is because cpuset_getid() for a pid wouldn't make sense then and I expect administrators to be mostly interested in managing groups of processes.

It's really two different goals that are being served by this api. You can think of the sets as more of an administrative tool. While the private thread mask is a tool for the application programmer. It just so happens that it was convenient under the hood to have it all managed in the same way.

Thanks,
Jeff


-Alfred

_______________________________________________
freebsd-arch@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: [PATCH 6/6] sched: disabled rt-bandwidth by default
    ... standards and previous Linux behaviour by default for something that it is ... monopolization" is OK to me - it still keeps runaway CPU loops (which ... RT tasks have always been debuggable by using a simple watchdog thread. ... us breaking our API to cater for their idiocity. ...
    (Linux-Kernel)
  • Re: [PATCH 6/6] sched: disabled rt-bandwidth by default
    ... I do not agree that it is too tight, ... monopolization" is OK to me - it still keeps runaway CPU loops (which ... RT tasks have always been debuggable by using a simple watchdog thread. ... us breaking our API to cater for their idiocity. ...
    (Linux-Kernel)
  • Re: yield API
    ... certain apps, i could immediately show you tons of examples where they ... just about any other core kernel concept or API. ... there is a need for something that will give up the CPU ...
    (Linux-Kernel)
  • Re: Linux compatible setaffinity.
    ... However, sometimes one needs a hard CPU binding interface like this one, and I don't see any reason to defer adding this interface in favor of the Apple one, since they are somewhat orthogonal. ... Of course, Jeff's implementation of the Linux API doesn't actually fully implement the API. ... the Solaris facility to be able to change a CPU set and have all the things pinned to it follow the centrally-administered set is a nice match for our concept of Jail. ...
    (freebsd-arch)
  • Re: eboot, accessing hardware
    ... If the code uses some OS API you may try to re-implement them inside ... the bootloader using the same prototype. ... If you provide more detail about the CPU you are using, ... (l'indirizzo di reply di questo messaggio non è ...
    (microsoft.public.windowsce.platbuilder)