Re: How do I send stdout to file and stderr to both file & tty?

paintedjazz@xxxxxxxxx wrote:
Many thanks but both of these commands don't quite do what I was hoping
for
since they both send stdout to the screen and I only want stdout to go
to the log file.
Stderr, on the other hand, should go to both or just the screen. Sorry
if I failed
to make that clear.

Simply put, redirect all verbiage into the file, but show the important
verbiage (error output) on standard output also, so that it shows up on
the tty, or can be separately redirected to another file if necessary.

In order to meet this requirement, is is necessary to open two separate
file descriptors, and connect them both to the log file. So that they
don't overwrite each other's output, they have to work in append mode.

Here is how:

The program's standard output needs to be duplicated into the standard
error position, so that error goes to the output.

First, we need to swap standard input and output. This has to be done
by a rotation through three descriptor positions:

command 3>&1 1>&2 2>&3 # x = y; y = z; z = x

Now, standard out is going to standard error, and standard error is
going to standard out!

So next, redirect standard error (which contains the output) into the
file, using append >>

command 2>> logfile 3>&1 1>&2 2>&3

Finally, use tee to also append the standard output (which contains
errors) to the logfile and show it as well:

command 2>> logfile 3>&1 1>&2 2>&3 | tee -a logfile

(We could use 2>&3- to close 3 after moving it to 2, to be a little
less sloppy).

Ah well, doesn't work for me.

The shell's >> and tee -a are not actually using proper append mode on
the file descriptor, and so are racing with each other to overwrite the
output. Apparently, append means open the file normally, but seek to
the end before writing.

It was worth a shot.

It feels liberating to just give up.

By the way, which one of the two is the culprit? I bet you it's tee.
I'm using bash. The documentation says that >> opens in append mode. We
can freaking prove it. One would be to run strace on bash while it runs
a script that appends, see if the O_APPEND mode is used. But that would
be boring, so let's be empiricists:

append ()
{
local i=0;
exec 3>> logfile
while [ $i -lt $1 ]; do
echo $2_$i >3;
i=$((i + 1));
done
}

append 10000 a & append 10000 b

In my resulting logfile, not a damn thing is missing. There are 10000
messages from both a, and b, with no missing numbers, and the thing is
consequently 22000 lines long.

It's that tee not appending like it should, piece of shit! So we have
to drop it from consideration, and declare ourselves stuck.

No wait, can we /write/ our own tee using the shell? You bet. All it
has to do is read standard input, which is easy: while read line ; do
.... done. And each line has to go to a file, and to standard output.
That's easy, we just need logfile opened in append mode, and whee,
right?

command 2>>logfile 3>&1 1>&2 2>&3 | (exec 3>>logfile ; while read line
; do echo $line ; echo $line >3 ; done )

Hmm, this doesn't quite seem to work for me, but this does:

command 2>>logfile 3>&1 1>&2 2>&3 | while read line ; do echo $line ;
echo $line >logfile ; done

Anyway, one finishing touch: create the error log at the beginning, or
truncate existing one to zero:

logfile ; command 2>> logfile ... etc

Not sure how portable that all is, but it seems to work with bash
3.00.16 on a Fedora Core 4 Linux box.

.

Relevant Pages

  • Re: How do I send stdout to file and stderr to both file & tty?
    ... they have to work in append mode. ... The program's standard output needs to be duplicated into the standard ... The shell's>> and tee -a are not actually using proper append mode on ...
    (comp.unix.shell)
  • Re: CDROM to flash drive -
    ... (I think it's in the standard ... repos) It's a bulk file renamer. ... you can append 01_ to the ...
    (Fedora)
  • Re: Isnt it time to PLACE it in the Forth 200x standard?
    ...  > any type of append e.g. ... belong in libraries, not the language standard. ... Core wordset. ...
    (comp.lang.forth)
  • Re: fopen(log_file, "a");
    ... >> the mode argument) causes all subsequent writes to the file ... >> open the same file for append, ... >> to the file without fear of destroying output being written ... The standard provides other assurances as far as reading back ...
    (comp.lang.c)
  • how to capture stdout/stderr only from child process?
    ... its own standard output and standard error (along with that of all its ... children) to a logfile while still sending it to the console. ... is a "controller" in the sense that it spawns a child process which does ... version the logging code is placed on the *child* side of the fork. ...
    (microsoft.public.win32.programmer.kernel)