Re: chopping off the last few lines.
From: Chris F.A. Johnson (cfajohnson_at_gmail.com)
Date: 04/22/05
- Next message: Chris F.A. Johnson: "Re: Selecting the best ftp mirror automatically."
- Previous message: William Park: "determining regular or array variable?"
- In reply to: didds: "chopping off the last few lines."
- Next in thread: Ed Morton: "Re: chopping off the last few lines."
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Fri, 22 Apr 2005 11:52:07 -0400
On Fri, 22 Apr 2005 at 08:54 GMT, didds wrote:
> I have variably sized (ie length differs) ascii data files arriving
> from an external source over which I have no control, and need to do
> various pre-processing actions upon it via a ksh script. The final
> part of this pre-processing is to chop off the final seven lines.
>
> The current solution is
>
> cat $FILENAME | grep -v DJDE | tail +15 | grep -v COPY | grep -v
> Filename | grep -v Inv | grep -v Report | grep -v ^L | grep . >
> /tmp/f1.${FILENAME}
>
> NL=`wc -l /tmp/f1.$1 | awk '{print $1}'`
> NL=`expr ${NL} - 7`
> head -${NL} /tmp/f1.${FILENAME} > <new filename>
> rm /tmp/f1.$FILENAME
>
> Now - this does the job quite nicely... BUT... its not exactly
> elegant. I'd rather avoid the use of a temp file as interim storage to
> begin with (not least because of the requirement to ensure it has a
> unique filename), so would prefer a continuation of the
> piped commands.
>
> However, I am not sure how I would do it as following the "grep ." I
> would need to pipe through to something like
>
> ... | head -`expr `wc -l | awk '{print $1}'` - 7`
>
> i.e. the parameter to pass to head would need to be within `` quotes,
> but THAT command would in itself ALSO need to use `` quotes - which
> won't work as I've written it!
>
> Has anybody any ideas how to embed evaluated commands (ie use of ``)
> within another evaluated command (ie within ``)?
>
> Or maybe a better way fo chopping a set number of lines from a variable
> length file?
This script removes a specified number of lines from the top and/or
bottom of a file. The only part you really need is the awk script
at the end:
## NAME: topntail
## USAGE: topntail [-b N] [-e N] [FILE ...]
b=1 ## No. of lines to remove from beginning
e=1 ## No. of lines to remove from end
## Parse command-line options
while getopts b:e: opt
do
case $opt in
b) b=$OPTARG ;; ## Number of lines to remove from beginning
e) e=$OPTARG ;; ## Number of lines to remove from end
esac
done
shift $(( $OPTIND - 1 ))
case $b$e in ## check for non-numeric characters in $b and $e
*[!0-9]*) exit 5 ;;
esac
if [ $e -eq 0 ]
then
sed "1,${b}d" "$@" ## just remove from the top
else
## The buf[] array is a rotating buffer which contains the last N lines
## where N is the number of lines to be removed from the bottom of the file
## Printing starts when the line number is equal to the sum of the number
## of lines to be removed from top and bottom, and continues to the end
## of the file; the earliest line in the buffer is printed.
## The last N lines will not be printed.
awk 'NR > b + e { print buf[ NR % e ] }
{ buf[ NR % e ] = $0 }' b=$b e=$e "$@"
fi
(This is based on scripts by Bill Marcum and Janis Papanagnou
posted here a few months ago, and appears in my forthcoming book,
"Shell Scripting Recipes" from Apress.)
--
Chris F.A. Johnson http://cfaj.freeshell.org/shell
===================================================================
My code (if any) in this post is copyright 2005, Chris F.A. Johnson
and may be copied under the terms of the GNU General Public License
- Next message: Chris F.A. Johnson: "Re: Selecting the best ftp mirror automatically."
- Previous message: William Park: "determining regular or array variable?"
- In reply to: didds: "chopping off the last few lines."
- Next in thread: Ed Morton: "Re: chopping off the last few lines."
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
