Re: deleting duplicate lines and ignoring whitespace
From: rakesh sharma (sharma__r_at_hotmail.com)
Date: 05/19/04
- Next message: guerrilla_thought: "Execute all processes in the background from bash."
- Previous message: Kevin Collins: "Re: Unix groups and file access"
- In reply to: Sam Heller: "deleting duplicate lines and ignoring whitespace"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: 18 May 2004 23:04:51 -0700
heller4@llnl.gov (Sam Heller) wrote in message news:
>
> I am familiar with the uniq program, but am trying to find a way to
> delete lines while ignoring embedded and external whitespaces.
> For example I would like the following:
>
> 123 4 567 8910
> 1 234567 8 910
> 1345
>
>
> to become
>
> 123 4 567 8910
> 1345
>
>
> It doesn't matter which of the duplicate lines is kept. The problem
> is that if someone put an extra space between columns or endedded with
> an extra space then the lines will be considered different, even
> though they contain the same data.
>
sed -e '$q;N;h;/^\(.*\)\n\1$/{g;D;};g;P;D' yourfile
- Next message: guerrilla_thought: "Execute all processes in the background from bash."
- Previous message: Kevin Collins: "Re: Unix groups and file access"
- In reply to: Sam Heller: "deleting duplicate lines and ignoring whitespace"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
