Re: finding consecutive repetitive lines



osiris@xxxxxxxxxx wrote:
On 22 Aug 2006 18:02:41 -0700, "Xicheng Jia" <xicheng@xxxxxxxxx> wrote:

Ed Morton wrote:
osiris@xxxxxxxxxx wrote:

I have 100+ files of about 150-200 lines each.
Each file has 8 aligned fields and could be separated with /usr/bin/cut.

After reading a line in a loop, I cut FIELD1, FIELD4 and awk FIELD5.
I then use FIELD1 to get the line that was just read and
the following 4 lines with:

/usr/local/bin/grep -A4 "^$FIELD1" "$FILE"

which I then pipe to grep with

|grep "${FIELD4}.*${FIELD5}" |wc -l

I assign the count to a variable
and then test the variable if is equal to 5.

I am getting some accurate results but not perfect results.

To clarify, I want all lines printed if $FIELD4 and $FIELD5
are the same in five successive lines.

Can anyone see the fault in my logic or is there a better way?
Thanks for any help.

Something like this (untested) should do it:

awk '{
key = $4 FS $5
if (key == prev) {
cnt++; saved = saved $0 ORS
if (cnt == 5) {
printf "%s", saved
cnt = 0; saved = ""
}
} else {
cnt = 0; saved = ""
}
prev = key
}' file

Think about what you want to do for the 6th consecutive matching line....

Ed.

A version which will print all records having a identical key within
more than 5 consecutive lines.

{
key = $4 FS $5
if (key == prev) {
cnt++; saved = saved $0 ORS
next;
} else if (cnt >= 5) {
printf "%s", saved;
}
cnt = 1; saved = $0 ORS
prev = key
}

--
XC

This worked almost perfectly. For some reason it skipped a known occurrence
of one the repetive fields that was at EOF. Would it be easy to also
include the line before and the line following (just to see the change)?
Thanks a bunch.

yes, you need to add a END block, like:

{
key = $4 FS $5
if (key == prev) {
cnt++; saved = saved $0 ORS
next;
} else if (cnt >= 5) {
printf "%s", saved;
}
cnt = 1; saved = $0 ORS
prev = key
}
END {
if (cnt >= 5) printf "%s", saved;
}

--
Xicheng

.



Relevant Pages

  • Re: finding consecutive repetitive lines
    ... Each file has 8 aligned fields and could be separated with /usr/bin/cut. ... After reading a line in a loop, I cut FIELD1, FIELD4 and awk FIELD5. ... if (key == prev) { ...
    (comp.unix.shell)
  • Re: finding consecutive repetitive lines
    ... After reading a line in a loop, I cut FIELD1, FIELD4 and awk FIELD5. ... if (key == prev) { ...
    (comp.unix.shell)
  • Re: finding consecutive repetitive lines
    ... After reading a line in a loop, I cut FIELD1, FIELD4 and awk FIELD5. ... if (key == prev) { ... Think about what you want to do for the 6th consecutive matching line.... ...
    (comp.unix.shell)
  • Re: How to read an environment variable?
    ... Your example doesn't call putenv() to set the environment variable ... You might also want to add a call to printf() that announces ... when envLibIinit() returns ERROR. ... Prev by Date: ...
    (comp.os.vxworks)
  • Re: Any easy to printf an interger in 9,999, 99 format?
    ... >Is there any easy way to printf an integer in a way like ... >I know "%d" can be usd to print it as 1234567. ... char *format64 ... Prev by Date: ...
    (comp.lang.c)