Re: Redirect stdout and stderr preserving the original order

2009-12-30, 18:21(-08), bsh:
[...]
Icarus Sparry wrote:
... For typical programs, this is very hard. The reason is that
almost all programs use the "stdio" (standard I/O) library.

This is why ksh93 utilizes the Kiem-Phong Vo's stdio emulation
library, sfio ("Safe and Fast I/O"). It is a component in AT&T's
AST library, which happens to be statically linked into ksh93,
which is why if the above doesn't work, to try to use this shell
instead. Pre-compiled version exist specifically for RHEL and many
other Unices.
[...]

In which way does that solve the problem. Are you saying that
commands run by those shells will use a different stdio by some
sort of LD_PRELOAD trick?


John Koy <John....@xxxxxxxxxxx> wrote:
All these tricks are useless. One cannot achieve synchronousness
across process boundaries without explicit IPC synchronization,
like using semaphores.

Yes and no. Since IRIX 3+, all the stdio functions and macroes
are written to preserve line-by-line serialization (in the old
sense of the term). Presumably other systems are written this
way, but I have no information. Like I indicated above, there
is a reason that sfio was created!

That has nothing to do with stdio here. With pipes, you'd need
some sort of syncronisation.

The problem is that you need to have a process read from 2
pipes, and you can't guarantee that this process will read the
data in the order they were written. You could only guarantee
that if writing to the pipe was blocking until the reading
process has read everything (and thanksfully, pipes don't work
like that) and if there aren't processes writing simultaneously
to stdout and stderr.

For instance, in

echo x
echo y >&2
echo z

which results in

write(1, "x\n", 2);
write(2, "y\n", 2);
write(1, "z\n", 2);

(echo doesn't use stdio, or fflushes afterwards if it does
anyway).

What is likely to happen is that that command will perform the 3
writes at once before any other process is scheduled.

Then your process that reads the other end of those 2 pipes will
read "x\nz\n" from the first one and "y\n" from the second in no
predictable order, and if it's 2 separate processes reading the
2 pipes, the result is even less likely to be predictable.

If indeed semaphores are your only resort, after ten years of
research and experimentation, I have come to the conclusion,
that at least until ksh has the builtin ability, the only decent
semaphore library and batch job frontend, are:

"semaphore.ksh"

Here you need semaphores in between write(2) and read(2) system
calls, so I don't expect a shell solution to be of any use.

a LD_PRELOAD solution with a modified stdio that would
do the logging could work assuming all the commands called by
the shell use a dynamically linked stdio..

--
Stéphane
.

Relevant Pages