Extract lines from a log-file with duplicate fields?

How could I extract all of the lines from a log-file which have a
certain field duplicated?

For example, if File1.log contained

1,1,Fixed
2,2,Geared
3,1,Indexed
4,4,Hub
5,4,Freewheel
6,4,STI

Using any common GNU utilities like sed/awk/uniq/tac/egrep (not Perl
though), let's say I need a command that will return all lines where
the second field is duplicated. Note that the duplicated field may or
may not be in consecutive lines. So, I need to end up with;

1,1,Fixed
3,1,Indexed
4,4,Hub
5,4,Freewheel
6,4,STI

For added points I could also do with a count of the number of times
the field was repeated (next to the field itself):

Field 1 repeated 2
Field 4 repeated 3

or even just:

1,2
4,3

Any thoughts?

Many thanks,

Duncan.

.

Relevant Pages

  • Re: Extract lines from a log-file with duplicate fields?
    ... DS> How could I extract all of the lines from a log-file which have a ... DS> the second field is duplicated. ... First sort on second field and then select lines which have that ...
    (comp.os.linux.misc)
  • Problems using Expect to parse a file
    ... I'm trying to write a script to automate a program called DeltaE, a powerful thermoacoustic design tool, and to extract performance data from the simulations it runs by parsing through its output files. ... Less, on the other hand, remains open upon reaching the end of a file until it receives the "q" command, so I tried writing a script to step through the file using that, as follows: ... The trouble now is, although manually executing the command "less -c data/1.out" at a tty functions just as you would expect it to, when Expect sends the exact same command as part of the spawn process, and running the script with exp_internal true, apparently none of the data sent to the display by less when started normally is being seen by Expect - all it seems to get is "No tags file\r\n", followed by eof and closure of Less - I'm afraid I haven't a clue what the "No tags file" statement means. ...
    (comp.lang.tcl)
  • Re: Extract single file from XP CD
    ... > about using MSCONFIG to extract a single file from the XP CD, ... (Command Prompt window opens) ... Type in these commands ... regsvr32 oleaut32.dll ...
    (microsoft.public.windowsxp.setup_deployment)
  • Re: Using GZip decompression in VB 6
    ... Decompress the gzip file. ... In the above commands /C after the cmd.exe means for the Command promtp ... Also, do not hardcode the path and name of the Command prompt EXE, if you ... then extract it. ...
    (comp.lang.basic.visual.misc)
  • Re: default stanza in AIX 5.2 /etc/security/limits?
    ... Sorry about the /tmp permissions issue. ... I usually extract to a directory like ... Maybe chsec would be a better command for default stanza, ...
    (AIX-L)