Re: coloring stdout and stderr

Christof Warlich <cwarlich@xxxxxx> wrote:
I'm writing a shell-like interpreter that processes commands. To make
its output more readabe, I'd like to color the output, i.e. I want that
everything printed to stdout shows up in green and everything printed to
stderr becomes red.

The (stripped down but still working) example shown below more or less
does what I want, particularly the coloring still works even if my shell
calls itself. But I had to make a few compromises:

1) For each command being executed by the shell, the sequence of
messages printed to stdout and stderr is broken: The shell will always
first print all (green) messages sent to stdout and then all (red)
messages sent to stderr. I tried pipes, but without much success, as
syncing with the color escapes soon became a nightmare.

Well, you have the program you invoke via system()_ write its
stdout to one file and its stderr to another and then you first
open one of the files and output what it contains and then the
you repeat this with the second file. How should it be possible
to restore the order of the original output when all infornation
about that is lost?

2) To allow that coloring works even when recursively calling the shell,
I had to redirect stdout and stderr to unused file descriptors (i.e 1>&7
and 2>&8).

Why?

Does anyone have an idea how to overcome these drawbacks? Here is the
example, my apologies as it is probably too long to be a good candidate
for being posted to ask for help, but I stripped it down to its very basics.

#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>
#include <string.h>
#include <unistd.h>

int main(void) {
FILE *out, *err, *prompt;
if(dup2(7, 7) == -1) // The stdout replacement.
out = fopen("/dev/tty", "wa");
else
out = fdopen(7, "wa");
if(dup2(8, 8) == -1) // The stderr replacement.
err = fopen("/dev/tty", "wa");
else
err = fdopen(8, "wa");
if(dup2(9, 9) == -1) // Any control characters go here.
prompt = fopen("/dev/tty", "wa");
else
prompt = fdopen(9, "wa");

What's that dup2()ing random (possibly not open) file descriptors
meant to be good for?

char *l = 0;
size_t n;
fprintf( prompt, "^[[01;34m$ ^[[0;;m^[[K" );
fflush( prompt );
while ( getline(&l, &n, stdin) != EOF ) {
*(l + strlen(l) - 1) = 0; // hack to strip \n
char outName[] = "outXXXXXX";
char errName[] = "errXXXXXX";
int outDesc = mkstemp(outName);
int errDesc = mkstemp(errName);

You could unlink() both files already at this moment. The
actual file only gets removed once you close() it, and this
way they aren't visible in the file system anymore and also
vanish if your program, for whatever reasons, should crash.

int commandSize = strlen(l) +
strlen(outName) +
strlen(errName) +
10;
if(n < commandSize) {

This will always be the case, since 'commandSize' will always
be longer than what 'l' is pointing to.

char *old = l;
l = (char *) realloc(l, commandSize);

Are you writing C++? If not, drop the cast, it doesn't help
a bit, it just will keep the compiler from warning you if
you forgot to include <stdlib.h>. And if you use C++ should-
n't you be using the new operator?

strcpy(l, old);

Why do you do that copy? realloc() guarantess that the content
of what 'old' pointed to is copied to what 'l' points to. Even
worse, if 'old' and 'l' are identical (since realloc() found
enough memory after 'old' to just extend the memory region) that
does an (useless) in-place copy, which isn't allowed with strcpy()
(only memmove() can be used in such situations).

n = commandSize;
}
sprintf(l, "%s > %s 2> %s", l, outName, errName);
system(l);
FILE *fp;
fputs("^[[01;32m", prompt);
fflush(prompt);
fp = fdopen(outDesc, "r");
while(getline(&l, &n, fp) != EOF) {
fputs(l, out);
}
unlink(outName);

You must also close() the file, otherwise you're going
to keep it open (though invisible in the file system)
until the program ends. And if you a lot of commands
then you could run out of file descriptors after some
time since all the old files are still open.

fflush(out);
fputs("^[[01;31m", prompt);
fflush(prompt);
fp = fdopen(errDesc, "r");
while(getline(&l, &n, fp) != EOF) {
fputs(l, err);
}
unlink(errName);

Same here, also should be close()ed.

fflush(err);
fprintf(prompt, "^[[01;34m$ ^[[0;;m^[[K");
fflush(prompt);
}
return 0;
}

Please note the escape characters for color switching, I'm not sure it
they survive the copy and paste if you want to compile the example.

They probably are better replaced by something portable,
ncurses comes to mind.

I guess the only chance to get all this (more or less) right is
to use pipes. In main() you create 2 pipes, one for stdout and
one for stderr. Now fork() and in the child process you close
the read ends of both pipes, then dup2() the other ends to stdin
and stdout. Then also close the write ends (they aren't needed
anymore). Then exec the program you want to run. Its writes to
stdin will now go to the parents read end of one of the pipes,
its writes to stderr to the other.

In the parent you close the write ends of both pipes. Then you
call select(), waiting for either of the pipes becoming readable.
When one of them becomes readable just read as much as possible
and output it to the screen after sending you color setting.
When reading after a successfull call of select() returns 0
then the other side has closed that pipe and you can stop
listening on it.

You should also install a signal handler for SIGCHLD to figure
out when the child exits.

There is one possible problem: If select() reports that both
pipes can be read from you can't determine which one came
first - the exec()ed programs write to stdout or the write
to stderr. There's nothing that can prevent that. I probably
would go for printing out what came from stderr first.

Now, a worse problem is that the program on the other side
(or the printf() subsystem it uses) will detect that it's
writing to a pipe and not a terminal. A pipe is handled
like a file in that full buffering gets switched on. For
output to a terminal only line buffering is used (i.e.
internal buffers are flushed when a '\n' is found in the
output). With full buffering printf() will wait until the
internal buffer is full. This concerns stdout only since
for stderr buffering is always switched off. The result is
that messages from the program to stderr will get send via
the pipe immediately, while writes to the pipe will be come
not in lines but in large chunks of data whenever the buffer
becomes full.

This can only be avoided if you use a pseudo-terminal instead
of a simple pipe for what the exec()ed program writes to its
stdout...

In case your program calls itself you have another (minor)
problem because then color settings will get output twice.
But since setting a color twice should make no difference
this probably can be disregarded.

Regards, Jens
--
\ Jens Thoms Toerring ___ jt@xxxxxxxxxxx
\__________________________ http://toerring.de
.

Relevant Pages

  • Re: coloring stdout and stderr
    ... I want that everything printed to stdout shows up in green and everything printed to stderr becomes red. ... I tried pipes, but without much success, as syncing with the color escapes soon became a nightmare. ... After each call to getline, n contains the size of the malloced buffer, not the lenght of the read line. ...
    (comp.unix.programmer)
  • Re: order of execution
    ... stdout and stderr are different streams. ... the standard error stream is not fully buffered; ... C defines three buffering modes for streams: ...
    (comp.lang.c)
  • Re: Process.BeginErrorReadLine and BeginOutputReadLine; buffer not flushed?
    ... script, and redirects the stdout and stderr output from that script ...  But it's not just the StreamReader that does buffering. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: correct way of running a sub-process
    ... misled by the name - the C library function ... You can turn off buffering, ... reading from two different pipes for stderr and stdout, ...
    (comp.lang.python)
  • Re: Using popen with 0,1,2,3,4 streams
    ... does what you need (since it returns an array if three pipes: stdin, stdout ... and stderr for the subprocess) but the code is short if a little arcane, ...
    (comp.lang.ruby)