Re: I need a custom script to extract data from my web logs.

From: Java Dude (Brian.Flanagan_at_scott.af.mil)
Date: 10/01/03 

Date: 1 Oct 2003 08:37:37 -0700

Brian.Flanagan@scott.af.mil (Java Dude) wrote in message news:<58bbdc08.0309302104.4a2d2cb7@posting.google.com>...
> re:
>
> > >>#!/usr/bin/sh
> > >>grep myfile.html $1 |
> > >> while read ip a b c d
> > >> do
> > >> {
> >
> > Where did the "{" and "}" come from?
>
> From various tests I was doing. They don't hurt anything.
>
> > >> host = `lookup $ip`;
> >
> > You can't put spaces around the "=" here. I didn't notice that before.
>
> I will remember to remove the spaces then!
>
> > >> awk '{ printf("%s %s %s %s %s %s \n",$a,$b,$c,$d,$ip,$host); }'
> >
> > You aren't passing anything to awk and you don't need awk anyway. The problem
> > you had with printf was just a shell syntax error - the line should've been:
> >
> > printf "%s %s %s %s %s %s \n" "$a" "$b" "$c" "$d" "$ip" "$host"
>
> Yeah. (Doh!) My original script was using awk to output the data by
> piping the output of the original grep through awk to the printf. I
> agree. It isn't needed. I'll fix the printf as you suggest!
>
> > <snip>
> > >
> > > I figured out part of the problem in doing it the way you suggested.
> > > When I read in the log this way, the entire line of the file goes into
> > > the variable "ip".
> > <snip>
> > That would only be true if your IFS was set to somthing odd or your grep returns
> > a line with no white-space.
>
> No. The line returned had white space. You saw the script (in the
> previous message) that returned the whole line into the ip variable.
>
> > Change your script to this:
> >
> > #!/usr/bin/sh
> > grep myfile.html $1 |
> > while read ip a b c d
> > do
> > host=`lookup "$ip"`;
> > printf "%s %s %s %s %s %s \n" "$a" "$b" "$c" "$d" "$ip" "$host"
> > done
> >
> > and see if you still have a problem. If you do, then post a sample of your input
> > file, highlighting a line you want the grep to pull out.
>
> I'll give those changes a try and let you know.
>
> Thanks!

O.K. I just have one more problem to solve.

For some reason the script doesn't want ot accept wildcards on the log
file name. I need to be able to feed it a series of logs to search
for the grep. (i.e. searchlogs log???? or searchlogs log*)

That's assuming the script we've been working with is called
"searchlogs".

Relevant Pages

  • Re: simple script that kills a process
    ... you can eliminate the grep while you are at it: ... ITYM: ... Never use single ticks to jumb back and forth between awk and shell as ... It looks like the script name is in the final field, ...
    (comp.unix.shell)
  • Re: I need a custom script to extract data from my web logs.
    ... > For some reason the script doesn't want ot accept wildcards on the log ... (i.e. searchlogs log???? ... wildcards, the shell expands these *before* starting the script, thus if ... If you want to pass all files to grep, use $* instead of $1 and see what's ...
    (comp.unix.shell)
  • Re: file name and line number
    ... grep having this depth of "print tuning". ... I already have a sed script I ... > just bite the bullet and use awk. ...
    (comp.unix.shell)
  • Shell Script to Parse Conf Files
    ... so I'm solicity help to construct a script to parse a ... I trying to write a sh script that will set environment variables from ... I think awk can also handle what I'm trying to do with grep. ...
    (comp.unix.programmer)
  • Re: Another newbie question - delete file based on date
    ... > I have a need to delete a group of files based on file name pattern ... I've been looking into grep and awk, ... If I was writting this into a script ...
    (comp.unix.shell)