Re: cpuset and affinity implementation


On Tue, 26 Feb 2008, Nick Evans wrote:

On Sun, 24 Feb 2008 00:40:31 -1000 (HST)
Jeff Roberson <jroberson@xxxxxxxxxxxxxx> wrote:

Please see:
http://people.freebsd.org/~jeff/cpuset.diff

This is unfortunately intertwined with ULE's new CPU selection algorithm
so that code is in the patch as well. Otherwise, this includes a simple,
ugly userland tool called cpuset and all of the kernel support required.
I have tested this by creating sets and subsets and modifying their cpu
masks under load. I'm able to dynamically reprovision without issue.

This doesn't have support for jails but the infrastructure is there. It
also fails to modify sets if it would leave threads without a valid cpu
to run on. I have not implemented a force option but it will be trivial
to do so. The initial cpu set is also created before we know all_cpus so
it's faked up with all cpus set for now.

I mostly want people to look at the interface in cpuset.h and make sure
they agree with it before I start polishing to commit. I'm fairly happy
with the way the syscall api looks now. The code itself ended up being
much more complicated than I'd hoped due to locking considerations. Try
not to look at cpuset_setproc() ;).

If you want to actually try the patch, here's a couple of neat things to
do with cpuset:

cpuset -l 0-4 /bin/sh

This creates a new group with a list (-l) of cpus 0-4 inclusive and runs
sh in it.

cpuset -g -p <sh pid>

This will get (-g) the mask of cpus pid (-p) is allowed to run on.

cpuset -l 0,2 -p <sh pid>

This will restrict sh to running on cpus 0, 2 while its group is still
allowed 0-4.

cpuset -l 0,2 -c -p <sh pid>

This will modify the cpuset (-c) that the sh belongs to.

cpuset -l 0-3 -s 1

This will modify the set (-s) that all threads are in by default to
contain the first 4 cpus leaving the rest idled.

Feedback is appreciated.


Jeff,

Is it currently, or will it eventually be possible to assign network threads
to different cores? Everything appears to be driven by pid, but at least
according to top all interrupt "processes" show as pid 12. Also, if
kern.sched.topology returns 0 is it safe to assume I'm not getting the
benefit of the topology distinction between packages vs cores?

I forgot to remove the topology sysctl. It is meaningless now. If your machine on boot say something like:
Cores per package: 2

Then the scheduler is aware of the layout.

As for binding interrupt/kernel threads, you need to get the tid. Then you can set a mask. er, except I guess I didn't make a -t tid argument for cpuset yet. I'll add that to the next patch.

Thanks,
Jeff


Nick

_______________________________________________
freebsd-arch@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: Windows 2003 Server with Multiple CPUs
    ... Hyperthreading capability displays two CPUs for every one. ... >I did modify the view and I have 4 windows representing 4 cpu's. ...
    (microsoft.public.windows.server.general)
  • Re: cpuset and affinity implementation
    ... If you want to actually try the patch, here's a couple of neat things to ... This creates a new group with a list of cpus 0-4 inclusive and runs ... This will get the mask of cpus pid is allowed to run on. ... This will modify the cpuset that the sh belongs to. ...
    (freebsd-arch)
  • Re: [patch] x86_64, irq: check remote IRR bit before migrating level triggered irq
    ... ioapic_level irq is limited, So if we keep vector number not changed ... when imgration to other cpus. ... it will need modify a little with assign_irq_vector and ...
    (Linux-Kernel)
  • cpuset and affinity implementation
    ... This is unfortunately intertwined with ULE's new CPU selection algorithm so that code is in the patch as well. ... Otherwise, this includes a simple, ugly userland tool called cpuset and all of the kernel support required. ... It also fails to modify sets if it would leave threads without a valid cpu to run on. ... The initial cpu set is also created before we know all_cpus so it's faked up with all cpus set for now. ...
    (freebsd-arch)
  • Re: [RFC PATCH] Dynamic sched domains aka Isolated cpusets
    ... * This is perhaps the first non-trivial cpuset patch to come ... set of CPUs isolated in child cpusets, ... This same issue, in a strange way, comes up on the memory side, ... to partition the system, along cpuset boundaries. ...
    (Linux-Kernel)