Re: Choosing between ASTs and Threads

From: Bill Todd (billtodd_at_metrocast.net)
Date: 09/16/03 

Date: Tue, 16 Sep 2003 17:48:56 -0400

"Richard Maher" <maher_rj@hotspamnotmail.com> wrote in message
news:bk7s2t$k3t$1@titan.btinternet.com...
> Hi Vince,
>
> Let's cut to the chase; IMHO threads on VMS are *only* ever useful
when
> porting UNIX code based on a threaded architecture.

While I haven't used VMS since long before threads existed there, it sounds
here as if you're completely ignoring the issues of multi-processor
systems - even though you explicitly quote that concern just below.

...

> > If I read it right, a program designed around ASTs cannot make use of
> > more than one CPU at a time
>
> Bollocks! This is the sort of lie (or at best half-truth) spun by the ADA
> worshipping Larry Kilgallens of this world who know full well that in a
> True Master/Slave scenario (for example), the AST driven Master process
can
> delegate work to any number of worker processes (1:1 Kernel Threads) to
> achieve all of the benefits of parallelism and Symmetric multi-processing
> that they espouse.

It didn't sound as if the original poster wished to delegate work like that
(and of course such delegation has significant overheads of its own): he
appeared to want to make use of multiple processors within the activities of
a single process.

...

> You do not tell us much about what your application/problem is, so
for
> argument's sake I will assume that you want to have thousands of
network
> connections to a VMS server application.

You assume a workload that may lend itself to multi-process execution
(because the overheads of delegation pale beside the overheads of the
activities themselves - and those activities can be prosecuted largely
independently by the multiple server processes without noticeable
interaction between them or with the delegating parent).

What if the workload *doesn't* meet those assumptions? What if the
delegation cost *is* important and/or the multiple concurrent server
activities *do* need to coordinate closely among themselves (and hence
desire the efficiency of doing so within a single process rather than
between processes)? The certainly sounds like the kind of workload the
original poster was trying to deal with (and if not, it's certainly the kind
of workload *some* people have to deal with, hence is still interesting).

  A common way of solving this
> problem is to have a single "Communication Server" process that handles
all
> of the Network I/O and Authentication and a variable number of
"Execution
> Server" processes that perform the application specific functions
(database
> I/O etc). This pool of "Execution Server" processes can grow and shrink
as
> user demand requires in accordance with parameters set up by the
System
> Manager.
>
> The communication server is obviously AST threaded and must perform
*all*
> network I/O, timeouts and server management asynchronously. The
*absolute
> beauty* of this is that as long as you have your alignment
and/or
> granularity settings correct then there is *no* need to lock or
mutex
> around memory accesses. As long as all processing is done at AST level
with
> the mainline doing something equivalent to SYS$HIBER, VMS will ensure
that
> no AST (of similar access mode) will interrupt another AST, so let VMS
just
> queue them up on a first come first served basis. (The demarcation of
AST
> functionality at different access modes isn't that much harder.)

This is very similar to the way spinlocks are typically handled in a
single-processor kernel: the processor priority is simply raised on the
spinlock request to a level that won't allow another spinlock request to be
made, and lowered when the spinlock is released. The problem, once again,
is that the mechanism doesn't allow one to capitalize upon additional
processors when they are present to perform the work in parallel: by
definition, the process is executing only one AST at a time.

...

> Now, just try and have thousands of threads in a single process trying
to
> achieve the same functionality.

Some people do that, but (as I explained in my earlier answer) it's far from
optimal (too much unnecessary contention). So, by the way, is your
suggested solution above, unless your 'execution server' processes are
limited in number to no more than the number of CPUs expected to be
available. But then they may *under*-utilize the available CPU resources
unless they themselves execute asynchronously to keep busy - at which point
any apparent advantage in simplicity over the model I advocated disappears.

 Mutext here; Lock there; Just to add 1 to
> active_link_count.

The same objection applies to your suggested solution - and those
independent server processes are far less efficient in communicating with
each other than an equal number of threads within a single process would be.

  "But we've only got 16 (or 32?) kernel threads/process
> all the rest are just bull***?" Well then why don't you try that
other
> UNIX joke and FORK off a separate process for each one of your thousands
of
> users and see how that scales!

As I noted earlier, the optimal solution is to have a number of threads
equal to the expected number of available processors, and to operate
asynchronously within each such thread to keep them all busy. This does
require some modest programming to ensure that they never wait for
*anything* for any significant period - e.g., you need to implement simple
lock and mutex mechanisms which return to the thread instead of quiescing it
(after setting up structures which will place an appropriate item on your
work queue when the requested resource becomes available).

>
> In closing Vince, if you want more information then please ask away
and
> provide a bit more detail about what you actually want to do. You may
also
> want to have a look at Windows2000 and why Microsoft moved away
from
> threads and re-embraced Asynchronous Callbacks.
>
> THREADS JUST WON'T SCALE!!!

I think you may be confused about the reasons for (and benefits of) threads.
Not only can they be easier to work with when performance doesn't matter,
but they allow use of multiple processors within a single process (while
avoiding the overheads of multi-process interactions).

- bill