Re: [Long] about ksh93 (Was: Bourne Shell Programming on Windows)

From: Dan Mercer (dmercer_at_mn.rr.com)
Date: 06/27/03 

Date: Fri, 27 Jun 2003 02:56:55 GMT

"Stephane CHAZELAS" <stephane_chazelas@yahoo.fr> wrote in message news:slrnbfmhel.7oe.stephane_chazelas@pcchazelas.free.fr...
: Dan Mercer wrote:
: [...]
: >: Are you saying that David Korn is an unitiated or ignorant ksh
: >: programmer? I wouldn't have dared ;).
: >:
: >: He wrote:
: >: getopts '[-]' opt --???man
: >
: > In a man page, though my man page says "--??man" and "--man" works.
: > Again, this is a problem with any globbing. Also note that getopts --man
: > won't be called in a script.
:
: The code above is taken from
: http://www.research.att.com/~dgk/ksh/script/env
: a script made by David Korn, put on his home page (maybe to
: demonstrate the /power/ of ksh93).

It's a sample. It even contains commented out code. His computer
has an env command.

:
: [...]
: > It's the way I've done it for 12 years. And certainly it's been mentioned
: > in this group a hundred times (at least 50 by me)
:
: Yes, another way is
:
: set -- [*] *
: case "$1$2" in
: "[*]*") ;;
: *) shift; for i
: do ...
: done;;
: esac
:
: Almost as legible ;).
:
: [...]
: >: 3. The newtest ([[ ... ]]) operator -a file is obsolete.
: >: Use -e instead.
: [...]
: > Yeah, I can never keep straight those two and I was too lazy to look it up.
: >
: >:
: >: Although I agree it was more accurate. It doesn't check for
: >: file existance, but for file accessibility.
: >
: > No, it tests for existance. It returns true even if the file has
: > no permission bits.
:
: By accessibility, I meant the ability to at least stat the file.
: You can actually list a directory content without being able to
: stat the files. If the file is not "stat"able (you don't have
: "x" permission on the directory for instance), [[ -a file ]]
: will return "false", that was not in the sense of access(2).
:
: But to list the content of the current directory, you must have
: "x" permission on it, so that doesn't apply in this very
: example.
:
: Only in example like:
:
: $ mkdir 1; touch 1/{a,b,c}; chmod 444 1
: $ echo 1/*
: 1/* 1/a 1/b 1/c
: $ for f in 1/*; do [[ -e $f ]] && echo "$f"; done
: $ set -- 1/[*] 1/*;case "$1$2" in "1/[*]1/*");;*)shift;for i;do echo "$i";done;esac
: 1/*
: 1/a
: 1/b
: 1/c
:
: [...]
: >: for file in *(N); do
: >: ...
: >: done
: >:
: >
: > Again, this is not a major problem and one easily dealt with, as are all error
: > conditions.
:
: I consider it is, as you get no warning. Your script will work
: for a while, and because of the ambiguity of "patterns" (they
: are no more patterns when they match nothing) you may run into
: troubles.
:
: [...]
: >: I provided one (courtesy of David Korn).
: >
: > You provided one obscure example that does not occur in scripts.
:
: I provided an example from a script by David Korn.
:
: > Show an example that might occur in a script.
:
: A very common example is the:
:
: find /dir -name *.html -print
:
: We see sometimes in this NG.
:
: If there's no html file in the current directory, it will work
: OK. if there's one file in the current directory, it will (without
: any error or warning) return an incorrect result, if there are
: more, it will return an error.

So somebody shoots themselves in the foot. You can do that with anything.

:
: [...]
: > No, "time" and "!" do not take arguments, any more than "for" or "while"
: > do. "time" runs the pipeline following it and displays the timing for it. If it were
: > a builtin, it would only run the first sequence in the pipeline. The same thing goes for
: > ! - if it were not a keyword, it would only reverse the truth of the first segment
: > of a pipeline, not the whole pipeline.
:
: Yes, you know, I know why "!" can't be a builtin. Who else? For
: time, there's a "/usr/bin/time" command, if you don't read the
: ksh man page in detail or issue a "type time", you may think
: it's a command or builtin.

Again, DK is not responsible for the problems of people who won't
read a man page. You can't do anything in zsh without reading a dozen\man files or perl
without reading a book.

:
: And what about the typeset/export builtins/keywords?
:
: But I fear we're nitpicking here.
:
: [...]
: >: Other examples from Mr Korn. Again in its "env" script.
: >: To clear the environment "typeset +x $(typeset +x)"
: >: It only clears shell variable like environment variables, and
: >: not the "_" one (well it's still exported to the next command
: >: run).
: >
: > No, you are wrong on this. The sequence above clears the export
: > flag. But "$_" is one of the inherent variables, like "$!" "$$" "$PPID"
: > in every shell. It is automatically marked for export (why, I don't know).
: >
: > So again, this is a misunderstanding on your part, not a failure on Korn's.
:
: There are two things about $_. First, the $_ *shell* variable
: (not exported) is the last argument of the last command run.
sometimes.
:
: Then, ksh passes a "_" environment variable to every process it
: execs containing the path of the program.
:
: _=1 env
: returns "_=/usr/bin/env"
:
: In zsh, "_" is readonly, you can't modify or unset it. But
: that's also nitpicking.
:
: >: Another one in the same file:
: >:
: >: command unset $OPTARG 2> /dev/null
: >:
: >
: > Korn's example are just that - quick and dirty examples
:
: If there's somewhere where you have not to be dirty, it's in
: example. An example is something intended to be reproduced.

Yeah, but who's going to try to unset a variable with an invalid
name. GIGO applies there.
:
: > not full
: > fledged production code. The example file you give even has a commented
: > out line of code. Since the globbing chars do not form legitimate
: > variable names, it's unlikely you would be trying to unset a variable
: > with that name.
:
: They are legitimate environment variable names (and because they
: are not legitimate *shell* variable name, one may want to unset
: them).

I don't think ksh allows invalid env names. You can use env to
create them for dumb programs that like to break the rules, but
I don't remember if ksh supported them. I'd check, but my
noisy Linux machine is shut down,

:
: But my point was that even the most experienced shell programmer
: can't help making those kind of errors because of the nature of
: ksh syntaxe and because it is too much permissive.
:
: [...]
: >: >: Each time you want to split a string, you have to disable filename
: >: >: generation (on by default).
: >: >
: >: > Again, I need to see an example of something broken.
: >:
: >: For example, one of your posts:
: >: Message-ID: <Cm6Fa.2548$fe.16173@twister.rdc-kc.rr.com>
: [...]
: > So you change the above to:
: >
: > function split {
: > set -f
: [...]
:
: Yes, that was exactly my point, the point you needed an example
: for.
:
: [...]
: > The actual split function I employed at work is far longer and
: > more bullet proof.
:
: And how much time did you spend writing it. If you had been
: using perl, awk, python, there was already a function to do it.
: I'd be curious to see your bullet proof split command. How does
: it deal with IFS white spaces?

Well, perl and awk started out as text processors. That isn't
what the shell is all about.
:
: [...]
: > If I want consistent typing, I'll write in a higher level language. Note that
: > the only variable type in UNIX is a string, regardless of its contents.
:
: environment variables you mean? ksh have several variable types

That's what I intended to type.

: (string/array/patter, integers, floating point, hashes...)

But it's not really typing. It's much looser than that.

: as you know.
:
: [...]
: >: >: 5- command substitution removes too many NLs
: [...]
: > But then how would you deal with all the unwanted newlines?
:
: Do you have an example where there are additional unwanted
: newlines beside the trailing one?

That is the unwanted one.
:
: Anyway, to remove trailing newlines:
:
: var=${var%"${var##*[!
: ]}"}

Which you would have to do after almost all command substitutions.

if [[ "$(somecmd)" = somevalue ]]

would become really ugly.
:
: > it's a design decision. All languages employ them.
:
: Not the good one, to my mind. But note that I know every shell
: works like that, that's not a ksh only issue.
:
: [...]
: > If by CGI you mean Common Gateway Interface, I've written plenty
: > of bullet-proofed cgi scripts in ksh. We chose ksh over perl because
: > there are a lot more people who can read ksh than perl. Perl would
: > have been my preference because of the modules. C'est la...
:
: Can you guarantee it's bullet proof?

Well, our test group was pretty inventive and they hammered them pretty hard.

:
: David Korn also has a
: http://www.research.att.com/~dgk/ksh/script/cgi-lib.ksh
:
: I didn't read it in much details, but it's likely that, by
: positionning some specific "Cookie" header, one may be able to
: know the content of any directory on the server:
:
: function cgi_cookie
: {
: typeset cookie=$1 name val c IFS=';'
: set -- $cookie
: [...]
:
: He forgot the "set -f".
:
: [...]
: >: >: In a shell, most forget to do
: >: >: cd -- "$var" || exit
: >
: > I don't think anyone would ever write that. You mean you'd want to
: > exit the shell if your cd failed. That could be tiresome.
:
: Maybe not exiting the shell, but at least doesn't continue as if
: you had managed to go in that directory of course (as the
: remaining of the script assumes "$var" is the new current
: directory).
:
: [...]
: >: Yes, but you and David Korn and I and every one fall in that
: >: trap.
: >
: > Actually I don't. If I'm changing to a directory I don't know exists I
: >
: > cd -- "$dir" || fatal "Can't change to $dir"
:
: Note that "cd" already displays the error message.
:
: >
: > where fatal is an error routine that prints a message and exits.
:
: Great, that should be teached in every shell programming book.
: And every one should read a shell programming book before
: writing scripts. I fear it's not always the case.
:
: [...]
: >: Thanks, why? Because I try to imagine ways how to do programming
: >: with shells? I think I have to agree, that's a bit silly.
: >
: > No, because your arguments fly in the face of massive amounts of
: > experience. You find these extreme examples that in general
: > people don't experience (otherwise I think you would hear massive complaints).
:
: Yes because few people do shell programming for complex
: application, and so that it remains so, I prefer posting those
: kind of warnings when someone advertises shell programming.
:
: ksh93 has been quite unknown so far. As it is now released with
: sources, many might want to try it and beleive its claim for
: being a "powerful programming language".

Well, I've seen and done some pretty powerful things with it.

:
: [...]
: >: Not really, except in the case of setuid scripts (check the
: >: unix-faq). And you'll have problem with perl or python too,
: >: that's more a shebang issue than a shell issue.
: >
: > And how many systems support setuid scripts? For that matter, does
: > ksh support setuid scripts? Anyone fooling with setuid scripts has
: > already one foot deep in alligator infested waters.
:
: Agreed, and mostly because it's difficult to write reliable
: scripts.

I remember one setuid script that ssaved my bacon. I sat down at the
console to apply a patch tape for ksh and another to remove the setuid
script. Someone had stupidly changed root's shell to ksh. A bad SCSI
caused a panic before the ksh finished upgrading - I had only
half a ksh. You couldn't get in as root or get an su or change /etc/passwd.
But I could exploit the su script to give me a root sh and fix passwd.

:
: [...]
: >: Yes, same as above. In bash, you can do "help for".
: >
: > Or you can just read the man page. Again, nothing to do with
: > programming.
:
: But ksh man page is not complete (see the getopts description in
: the man page). And my post was not only about programming.
:
: [...]
: > ps -ef | while read -A Data;do USER[${#X[@]}=${Data[0]} ...;done
: >
: > and still have the USER array populated after the while makes it far superior.
: > coprocesses makes it far superior.
: > FPATH makes it far superior.
:
: ;) OK, my turn:
:
: ~/.bashrc makes bash superior (well, actually, zsh's way is much
: more consistent).
:
: Its builtin command line editor makes it far superior
:
: history expansion makes it far superior (well, I don't like nor
: use it, even in zsh, but one could say that).
:
: prompt facilities make it superior.
:
: programmable completion makes it superior
:
: PIPESTATUS makes it superior

Well, PIPESTATUS is the only one of those I would agree on.
However, even PIPESTATUS may not be reliable all the time.
If PROMPT_COMMAND is set, PIPESTATUS will be set
to the exit status of the PROMPT_COMMAND pipeline,
not the last typed command.
:
: Note that I use neither bash nor ksh93, as you probably have
: already guessed. I have to agree bash is not much better than
: ksh. Most of the argument I wrote also apply to bash.
:
: [...]
: > Only for ksh88. ksh93 only sources env for interactive shells.
:
: Right, I stand corrected. But ENV is still read by
: non-interactive POSIX compliant shells.
:
: [...]
: >: >: for i in 1 2
: >: >: {
: >: >: print -r -- "$i"
: >: >: }
: >: >:
: >: > cool
: >:
: >: Isn't it? It's in every Bourne like shell.
: >
: > And the problem is?
:
: Not really important. Just what I said in my original post.
:
: [...]
: >: And I didn't speak of all the issues related to LC_COLLATE, LC_CTIME...
: >: In other programming languages, you have to explicitely activate
: >: localization. In shells, you have to disable it.
: >
: > Again, not a problem for most people.
:
: In the US you mean? When is ksh93 ready for unicode?
:
: [...]
: >: >: 17- "export var" when var is unset doesn't put "var" in the environment.
: >: >
: >: > And it shouldn't. It only marks it for export. "export var=" will put it in the
: >: > environment.
: >:
: >: Yes, you're right even if that depends on how you interpret POSIX:
: >:
: >: The shell shall give the export attribute to the variables
: >: corresponding to the specified names, which shall cause them
: >: to be in the environment of subsequently executed commands.
: >:
: >: Most modern shells seem to agree with you.
: >
: > So the problem is?
:
: None, I just say you were right and I was wrong.
:
: [...]
: > Tried zsh. Didn't like it. It's one of those bathtub solutions - it tries
: > to do too much. It's simply way too complicated, like emacs. I had
: > a buddy who loved emacs, loved to say how it improved his
: > productivity. Good thing too, since he spent 6 hours a day
: > palying with it instead of doing his assigned work. When learning
: > about a shell takes more effort than it's worth, why bother?
:
: Even without reading the man page, you benefit from zsh. Its
: syntax, key binding is much intuitive, it's completion system
: powerful and easy to use. It's globbing facilities are worth
: learning.
:
: You may want to give zsh 4.0 a try. Completion is now mature.
:
: A question Dan, as you seem to know ksh93 quite well, is this a
: bug or feature:
:
: $ a=" ."
: $ printf "<%s>\n" ${a#' '}
: <>
: <.>
:
: I don't understand the result. ${a#' '}ought to be quoted.
I had a dtksh script, built on an old version of ksh93, that
used to crap out on one statement. If I put in a print statement,
it would work. If I did set -x, it would work. I finally put in
a ":" on the previous line and , voila, it worked. Now it's possible
I introduced the problem by adding my own builtins. The method
for agging builtins changed between dtksh and current ksh93's.

Dan Mercer
dmercer@mn.rr.com

I don't either. Probably a bug
:
: --
: Stéphane

Relevant Pages

  • Re: [Long] about ksh93 (Was: Bourne Shell Programming on Windows)
    ... Are you saying that David Korn is an unitiated or ignorant ksh ... > You provided one obscure example that does not occur in scripts. ... > in every shell. ... that should be teached in every shell programming book. ...
    (comp.unix.shell)
  • Re: [Long] about ksh93 (Was: Bourne Shell Programming on Windows)
    ... :> But this is a shell feature, not a shell programming feature. ... You provided one obscure example that does not occur in scripts. ... You have to perfectly know the syntax of ksh to be able to ...
    (comp.unix.shell)
  • Re: Ksh and the parent/child relationship - a deep dive
    ... Can someone smarter that me provide an example of a parent ksh script ... korn shell script",DF ... "Unix Review: Shell Corner: Managing Background Commands in Shell ...
    (comp.unix.shell)
  • Re: [Long] about ksh93 (Was: Bourne Shell Programming on Windows)
    ... :> Korn Shell 93 ROCKS! ... I consider ksh and especially ... ksh93 one of the worst shell ever written, and especilly, its claim to ... But this is a shell feature, not a shell programming feature. ...
    (comp.unix.shell)
  • Re: Bash script help
    ... >>What IS the difference between calling awk or perl in a bash script? ... >>They are both considered programming languages..... ... The other level is how to write shell scripts. ... than standard unix utilities, it should be included in this FAQ. ...
    (comp.unix.shell)