Re: allocating kernel threads from a pool
- From: "Maxim Yegorushkin" <maxim.yegorushkin@xxxxxxxxx>
- Date: 5 Oct 2006 00:42:22 -0700
railrulez@xxxxxxxxx wrote:
I am implementing a user-space threading library (in Linux) that uses
kernel level threads to do IO, so that a user-level thread's IO calls
will not block the entire program. For performance reasons, I want to
create an initial pool of kernel threads, and every time a user-level
thread needs to do IO, it wakes up one of the idle kernel threads to
do the work (and subsequently yields to other user-level threads)
My current implementation strategy is this:
1. During initialization in the main process, create many kernel
threads using clone (with CLONE_VM set), and have them spin in a
while(1) { usleep(LOTS_OF_TIME); } loop.
2. Whenever a user-level thread needs to do IO, it picks a idle child,
and sends it a signal to wake up, using kill(2)
3. The child wakes up (or rather, gets jostled out of bed), and in the
signal handler, it looks up some data structure to find what operation
is required of it and where to dump the output, does the IO, and goes
back to sleep.
Any particular reason why you are not using mutex + condition? Benefits
of using them may be:
1) The implementation is simple and robust: mutex + condition +
container.
2) There is no syscall (kill) involved while posting a command, when
there is no contention, which is ofthen the case.
3) There is not much you can do from within the signal handler. Here
you don't need signals at all.
Another aproach is to create a datagram socketpair(). All your worker
threads are blocked in read() syscall on the same end of the pair. When
another thread writes a pointer to a command to another end of the
socketpair only one of the waiting worker threads is woken up to read
the data. No signal handlers, no mutexes/conditions, can't go simpler.
I don't know if this is a good scheme in the first place, but in any
case, I cant find a clean way to implement this. The problem I'm facing
is that in step 3, I can't find a good way to give the child the
information it needs to do the IO (ie, what function -- read()?
write()? what arguments? etc.) in a clean way. I have two options:
If you do want to use signals, you could use sigqueue() instead of
kill() to pass a pointer to your data along with the signal. That
pointer may be a pointer to a command (command design pattern). The
worker thread will invoke the command. The command contains all the
information reuired to do its job, it only has to be invoked.
Take a look at boost::function and boost::bind:
http://boost.org/doc/html/function.html
http://boost.org/libs/bind/bind.html
#include <unistd.h>
#include <boost/function.hpp>
#include <boost/bind.hpp>
typedef boost::function<void()> cmd;
int main()
{
// create commands
cmd a(boost::bind(write, STDOUT_FILENO, "hello ", sizeof "hello " -
1));
cmd b(boost::bind(write, STDOUT_FILENO, "world\n", sizeof "world\n"
- 1));
// execute them later elsewhere
a();
b();
}
[]
I'd be grateful for any design suggestions, as performance is very
important to me.
If performance is paramount use mutex + condition then.
.
- Follow-Ups:
- Re: allocating kernel threads from a pool
- From: railrulez
- Re: allocating kernel threads from a pool
- From: William Ahern
- Re: allocating kernel threads from a pool
- From: railrulez
- Re: allocating kernel threads from a pool
- References:
- allocating kernel threads from a pool
- From: railrulez
- allocating kernel threads from a pool
- Prev by Date: Re: allocating kernel threads from a pool
- Next by Date: Named pipe discarding old data
- Previous by thread: Re: allocating kernel threads from a pool
- Next by thread: Re: allocating kernel threads from a pool
- Index(es):
Relevant Pages
|
