Re: [Long] about ksh93 (Was: Bourne Shell Programming on Windows)

From: Stephane CHAZELAS (stephane_chazelas_at_yahoo.fr)
Date: 06/26/03 

Date: 26 Jun 2003 19:17:36 GMT

Dan Mercer wrote:
[...]
>: Are you saying that David Korn is an unitiated or ignorant ksh
>: programmer? I wouldn't have dared ;).
>:
>: He wrote:
>: getopts '[-]' opt --???man
>
> In a man page, though my man page says "--??man" and "--man" works.
> Again, this is a problem with any globbing. Also note that getopts --man
> won't be called in a script.

The code above is taken from
http://www.research.att.com/~dgk/ksh/script/env
a script made by David Korn, put on his home page (maybe to
demonstrate the /power/ of ksh93).

[...]
> It's the way I've done it for 12 years. And certainly it's been mentioned
> in this group a hundred times (at least 50 by me)

Yes, another way is

set -- [*] *
case "$1$2" in
  "[*]*") ;;
  *) shift; for i
     do ...
     done;;
esac

Almost as legible ;).

[...]
>: 3. The newtest ([[ ... ]]) operator -a file is obsolete.
>: Use -e instead.
[...]
> Yeah, I can never keep straight those two and I was too lazy to look it up.
>
>:
>: Although I agree it was more accurate. It doesn't check for
>: file existance, but for file accessibility.
>
> No, it tests for existance. It returns true even if the file has
> no permission bits.

By accessibility, I meant the ability to at least stat the file.
You can actually list a directory content without being able to
stat the files. If the file is not "stat"able (you don't have
"x" permission on the directory for instance), [[ -a file ]]
will return "false", that was not in the sense of access(2).

But to list the content of the current directory, you must have
"x" permission on it, so that doesn't apply in this very
example.

Only in example like:

$ mkdir 1; touch 1/{a,b,c}; chmod 444 1
$ echo 1/*
1/* 1/a 1/b 1/c
$ for f in 1/*; do [[ -e $f ]] && echo "$f"; done
$ set -- 1/[*] 1/*;case "$1$2" in "1/[*]1/*");;*)shift;for i;do echo "$i";done;esac
1/*
1/a
1/b
1/c

[...]
>: for file in *(N); do
>: ...
>: done
>:
>
> Again, this is not a major problem and one easily dealt with, as are all error
> conditions.

I consider it is, as you get no warning. Your script will work
for a while, and because of the ambiguity of "patterns" (they
are no more patterns when they match nothing) you may run into
troubles.

[...]
>: I provided one (courtesy of David Korn).
>
> You provided one obscure example that does not occur in scripts.

I provided an example from a script by David Korn.

> Show an example that might occur in a script.

A very common example is the:

find /dir -name *.html -print

We see sometimes in this NG.

If there's no html file in the current directory, it will work
OK. if there's one file in the current directory, it will (without
any error or warning) return an incorrect result, if there are
more, it will return an error.

[...]
> No, "time" and "!" do not take arguments, any more than "for" or "while"
> do. "time" runs the pipeline following it and displays the timing for it. If it were
> a builtin, it would only run the first sequence in the pipeline. The same thing goes for
> ! - if it were not a keyword, it would only reverse the truth of the first segment
> of a pipeline, not the whole pipeline.

Yes, you know, I know why "!" can't be a builtin. Who else? For
time, there's a "/usr/bin/time" command, if you don't read the
ksh man page in detail or issue a "type time", you may think
it's a command or builtin.

And what about the typeset/export builtins/keywords?

But I fear we're nitpicking here.

[...]
>: Other examples from Mr Korn. Again in its "env" script.
>: To clear the environment "typeset +x $(typeset +x)"
>: It only clears shell variable like environment variables, and
>: not the "_" one (well it's still exported to the next command
>: run).
>
> No, you are wrong on this. The sequence above clears the export
> flag. But "$_" is one of the inherent variables, like "$!" "$$" "$PPID"
> in every shell. It is automatically marked for export (why, I don't know).
>
> So again, this is a misunderstanding on your part, not a failure on Korn's.

There are two things about $_. First, the $_ *shell* variable
(not exported) is the last argument of the last command run.

Then, ksh passes a "_" environment variable to every process it
execs containing the path of the program.

_=1 env
returns "_=/usr/bin/env"

In zsh, "_" is readonly, you can't modify or unset it. But
that's also nitpicking.

>: Another one in the same file:
>:
>: command unset $OPTARG 2> /dev/null
>:
>
> Korn's example are just that - quick and dirty examples

If there's somewhere where you have not to be dirty, it's in
example. An example is something intended to be reproduced.

> not full
> fledged production code. The example file you give even has a commented
> out line of code. Since the globbing chars do not form legitimate
> variable names, it's unlikely you would be trying to unset a variable
> with that name.

They are legitimate environment variable names (and because they
are not legitimate *shell* variable name, one may want to unset
them).

But my point was that even the most experienced shell programmer
can't help making those kind of errors because of the nature of
ksh syntaxe and because it is too much permissive.

[...]
>: >: Each time you want to split a string, you have to disable filename
>: >: generation (on by default).
>: >
>: > Again, I need to see an example of something broken.
>:
>: For example, one of your posts:
>: Message-ID: <Cm6Fa.2548$fe.16173@twister.rdc-kc.rr.com>
[...]
> So you change the above to:
>
> function split {
> set -f
[...]

Yes, that was exactly my point, the point you needed an example
for.

[...]
> The actual split function I employed at work is far longer and
> more bullet proof.

And how much time did you spend writing it. If you had been
using perl, awk, python, there was already a function to do it.
I'd be curious to see your bullet proof split command. How does
it deal with IFS white spaces?

[...]
> If I want consistent typing, I'll write in a higher level language. Note that
> the only variable type in UNIX is a string, regardless of its contents.

environment variables you mean? ksh have several variable types
(string/array/patter, integers, floating point, hashes...)
as you know.

[...]
>: >: 5- command substitution removes too many NLs
[...]
> But then how would you deal with all the unwanted newlines?

Do you have an example where there are additional unwanted
newlines beside the trailing one?

Anyway, to remove trailing newlines:

var=${var%"${var##*[!
]}"}

> it's a design decision. All languages employ them.

Not the good one, to my mind. But note that I know every shell
works like that, that's not a ksh only issue.

[...]
> If by CGI you mean Common Gateway Interface, I've written plenty
> of bullet-proofed cgi scripts in ksh. We chose ksh over perl because
> there are a lot more people who can read ksh than perl. Perl would
> have been my preference because of the modules. C'est la...

Can you guarantee it's bullet proof?

David Korn also has a
http://www.research.att.com/~dgk/ksh/script/cgi-lib.ksh

I didn't read it in much details, but it's likely that, by
positionning some specific "Cookie" header, one may be able to
know the content of any directory on the server:

function cgi_cookie
{
        typeset cookie=$1 name val c IFS=';'
        set -- $cookie
[...]

He forgot the "set -f".

[...]
>: >: In a shell, most forget to do
>: >: cd -- "$var" || exit
>
> I don't think anyone would ever write that. You mean you'd want to
> exit the shell if your cd failed. That could be tiresome.

Maybe not exiting the shell, but at least doesn't continue as if
you had managed to go in that directory of course (as the
remaining of the script assumes "$var" is the new current
directory).

[...]
>: Yes, but you and David Korn and I and every one fall in that
>: trap.
>
> Actually I don't. If I'm changing to a directory I don't know exists I
>
> cd -- "$dir" || fatal "Can't change to $dir"

Note that "cd" already displays the error message.

>
> where fatal is an error routine that prints a message and exits.

Great, that should be teached in every shell programming book.
And every one should read a shell programming book before
writing scripts. I fear it's not always the case.

[...]
>: Thanks, why? Because I try to imagine ways how to do programming
>: with shells? I think I have to agree, that's a bit silly.
>
> No, because your arguments fly in the face of massive amounts of
> experience. You find these extreme examples that in general
> people don't experience (otherwise I think you would hear massive complaints).

Yes because few people do shell programming for complex
application, and so that it remains so, I prefer posting those
kind of warnings when someone advertises shell programming.

ksh93 has been quite unknown so far. As it is now released with
sources, many might want to try it and beleive its claim for
being a "powerful programming language".

[...]
>: Not really, except in the case of setuid scripts (check the
>: unix-faq). And you'll have problem with perl or python too,
>: that's more a shebang issue than a shell issue.
>
> And how many systems support setuid scripts? For that matter, does
> ksh support setuid scripts? Anyone fooling with setuid scripts has
> already one foot deep in alligator infested waters.

Agreed, and mostly because it's difficult to write reliable
scripts.

[...]
>: Yes, same as above. In bash, you can do "help for".
>
> Or you can just read the man page. Again, nothing to do with
> programming.

But ksh man page is not complete (see the getopts description in
the man page). And my post was not only about programming.

[...]
> ps -ef | while read -A Data;do USER[${#X[@]}=${Data[0]} ...;done
>
> and still have the USER array populated after the while makes it far superior.
> coprocesses makes it far superior.
> FPATH makes it far superior.

;) OK, my turn:

~/.bashrc makes bash superior (well, actually, zsh's way is much
more consistent).

Its builtin command line editor makes it far superior

history expansion makes it far superior (well, I don't like nor
use it, even in zsh, but one could say that).

prompt facilities make it superior.

programmable completion makes it superior

PIPESTATUS makes it superior

Note that I use neither bash nor ksh93, as you probably have
already guessed. I have to agree bash is not much better than
ksh. Most of the argument I wrote also apply to bash.

[...]
> Only for ksh88. ksh93 only sources env for interactive shells.

Right, I stand corrected. But ENV is still read by
non-interactive POSIX compliant shells.

[...]
>: >: for i in 1 2
>: >: {
>: >: print -r -- "$i"
>: >: }
>: >:
>: > cool
>:
>: Isn't it? It's in every Bourne like shell.
>
> And the problem is?

Not really important. Just what I said in my original post.

[...]
>: And I didn't speak of all the issues related to LC_COLLATE, LC_CTIME...
>: In other programming languages, you have to explicitely activate
>: localization. In shells, you have to disable it.
>
> Again, not a problem for most people.

In the US you mean? When is ksh93 ready for unicode?

[...]
>: >: 17- "export var" when var is unset doesn't put "var" in the environment.
>: >
>: > And it shouldn't. It only marks it for export. "export var=" will put it in the
>: > environment.
>:
>: Yes, you're right even if that depends on how you interpret POSIX:
>:
>: The shell shall give the export attribute to the variables
>: corresponding to the specified names, which shall cause them
>: to be in the environment of subsequently executed commands.
>:
>: Most modern shells seem to agree with you.
>
> So the problem is?

None, I just say you were right and I was wrong.

[...]
> Tried zsh. Didn't like it. It's one of those bathtub solutions - it tries
> to do too much. It's simply way too complicated, like emacs. I had
> a buddy who loved emacs, loved to say how it improved his
> productivity. Good thing too, since he spent 6 hours a day
> palying with it instead of doing his assigned work. When learning
> about a shell takes more effort than it's worth, why bother?

Even without reading the man page, you benefit from zsh. Its
syntax, key binding is much intuitive, it's completion system
powerful and easy to use. It's globbing facilities are worth
learning.

You may want to give zsh 4.0 a try. Completion is now mature.

A question Dan, as you seem to know ksh93 quite well, is this a
bug or feature:

$ a=" ."
$ printf "<%s>\n" ${a#' '}
<>
<.>

I don't understand the result. 

-- 
Stéphane

Relevant Pages
Loading