Re: Linux compatible setaffinity.


On Wed, 20 Feb 2008, Jeff Roberson wrote:

So perhaps this means a slightly more complex API, but not much more complex. How about:

int cpuaffinity_get(scope, id, length, mask)
int cpuaffinity_getmax(scope, id, length, mask)
int cpuaffinity_set(scope, id, length, mask)
int cpuaffinity_setmax(scope, id, length, mask)

Scope would be something on the order of process (representing individual processes or process groups, potentially), id would be the id in that scope namespace, length and mask would be as you propose. You could imagine adding a further field to indicate whether it's the current affinity or the maximum affinity, but I'm not sure the details matter all that much. Here might be some application logic, though:

Well I'm not sure about the max. How about just a cpuaffinity_get with a scope that specifies what cpus are available to you? If the set is restricted by a jail or some other mechanism it would be returned in avail. Otherwise all cpus would be returned. The thread probably wouldn't directly mainpulate its max, rather it would be set by changing the jail or cpu group it belonged to.

I think the actual details don't matter too much as long as we can express what we need to, so I'm ok with a special scope for that. You do raise an interesting point, though, on the nature of scope: presumably we'd like to be able to query a few different "maximums" in the interest of having a maximally debuggable system -- be it the hardware limit, the administrative limit, etc. So perhaps the scopes are something more like:

#define CPUAFF_SCOPE_HARDWARE 1 /* Hardware limits */
#define CPUAFF_SCOPE_SYSTEM 2 /* System usage mask */
#define CPUAFF_SCOPE_PROCESS 3 /* Processes and process groups */
#define CPUAFF_SCOPE_THREAD 4 /* Threads in process */

CPUAFF_SCOPE_HARDWARE would be what the kernel has probed at the physical layer. This would be get-only, and available simply to provide a consistent interface when dealing with CPU masks.

CPUAFF_SCOPE_SYSTEM would be what are exposed by the kernel for use by the application. This would be get-only without privilege, and presumably not exceed CPUAFF_SCOPE_HARDWARE. This would be the practical upper bound on what a process could use, and would likely be how we implement Jail scoping of CPUs. This might or might not be how one tries to discourage userspace use of a particular CPU.

CPUAFF_SCOPE_PROCESS would be the process affinity for the process as a whole. It would be get-set without privilege, but limited to CPUAFF_SCOPE_SYSTEM. I'm not sure we want that limitation to be something overridden with privilege, but I guess we can think about that.

CPUAFF_SCOPE_THREAD would be the thread affinity for an individual thread. It would be get-set without privilege. Do we limit it to CPUAFF_SCOPE_SYSTEM or CPUAFF_SCOPE_PROCESS?

Since this involves some access control now, a pondering on access control:

When a process sets the affinity of another thread or process, is it limited to the system scope of the current process, or of the target process? This may be a practical question if we're talking about how to deal with a process outside of a jail setting the affinity of a process inside jail. It's easy to implement the right thing in the kernel, but does that then imply that the system scope query should take a process ID so that a user tool can figure out what valid choices are? :-)

I think it's useful to play out a few of these scenarios a bit and see what works; we won't be locked into any particular model until it hits a -STABLE branch, but getting the underlying primitive right will make it a lot easier to implement some of the more mature services we have in mind, making them more likely to happen.

Robert N M Watson
Computer Laboratory
University of Cambridge
_______________________________________________
freebsd-arch@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • [PATCH] bitmap parsing routines, version 3
    ... int bitmap_weight(const unsigned long *bitmap, ... * bitmap_snprintf - convert bitmap to a hex string. ... * and a length, describing the mask contents. ...
    (Linux-Kernel)
  • [PATCH 03/18] flag parameters: paccept
    ... This syscall differs from accept in that it adds ... The flags parameter can be used to set flag like SOCK_CLOEXEC. ... we really want the signal mask ... handler (int s) ...
    (Linux-Kernel)
  • [PATCH 02/18] flag parameters: paccept
    ... This syscall differs from accept in that it adds ... The flags parameter can be used to set flag like SOCK_CLOEXEC. ... we really want the signal mask ... handler (int s) ...
    (Linux-Kernel)
  • [RFC] bitmap relative operator for mempolicy extensions
    ... mask is set, then the m-th bit of the destination mask will ... int nodelist_scnprintfFormat nodemask as list for printing ... +#define nodes_relative(dst, orig, relmap) \ ... * Set the n-th bit of @dst iff there exists some m such that the ...
    (Linux-Kernel)
  • Re: [PATCH 2/3] RT: Cache cpus_allowed weight for optimizing migration
    ... weight of the task->cpus_allowed mask. ... unsigned int time_slice; ... #ifdef CONFIG_PREEMPT_RCU ... This would properly update the migration state. ...
    (Linux-Kernel)