Re: print one occurance for duplicate entries

From: Harry (harryooopotter_at_hotmail.co_)
Date: 11/29/05 

Date: Tue, 29 Nov 2005 07:59:56 GMT

Ed Morton wrote...

>but almost all solutions that use getline are bad awk style at best or,
>more commonly, dangerously buggy or just plain wrong. Try this instead:
>
>awk '/dcm/{t=$0;next}!($0 in f){f[$0];print t"\n"$0}' file
>
>If that doesn't work, you'd be much better off trying to fix that (e.g.
>by providing more information here) than trying to make your posted
>script above work as it's starting way off on the wrong track.

Thanks Ed. It's getting better.

See below a portion of input and corresponding output.

>From the output, line 8 and line 10 are still duplicate (in terms of
the 6 values within the quotes), so are line 12 and 14.
The only different between line 8 and 10 are their 1st field different,
which should be ignored.

-- output (cat -n output) -- begin --
     1 ./2/ImageDatabase/w4028320/view0114.dcm
     2 000005A6: (0020,0037) [DS/60] "0.994367\-7.77195e-09\0.105995\0.
0229776\0.976221\-0.215558 " ;
     3 ./2/ImageDatabase/w4028320/view0152.dcm
     4 000005A0: (0020,0037) [DS/58] "-0.440021\0.89763\0.0253318\-
0.0137912\0.0214511\-0.999675" ;
     5 ./2/ImageDatabase/w4028320/view0170.dcm
     6 000005AA: (0020,0037) [DS/62] "0.999905\-1.11616e-10\-0.0137944\0.
000295905\0.99977\0.021449 " ;
     7 ./2/ImageDatabase/w4028320/view0210.dcm
     8 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
     9 ./2/ImageDatabase/w4028320/view0221.dcm
    10 000005A8: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    11 ./2/ImageDatabase/w4089355/view0007.dcm
    12 0000062A: (0020,0037) [DS/30] "1\-0\-0\0\-0.052336\-0.998629 "
;
    13 ./2/ImageDatabase/w4089355/view0010.dcm
    14 0000062E: (0020,0037) [DS/30] "1\-0\-0\0\-0.052336\-0.998629 "
;
     ;
-- output (cat -n output) -- end --

-- portion of input (cat -n input) -- begin --
      1 ./2/ImageDatabase/w4028320/view0114.dcm
     2 000005A6: (0020,0037) [DS/60] "0.994367\-7.77195e-09\0.105995\0.
0229776\0.976221\-0.215558 " ;
     3 ./2/ImageDatabase/w4028320/view0152.dcm
     4 000005A0: (0020,0037) [DS/58] "-0.440021\0.89763\0.0253318\-
0.0137912\0.0214511\-0.999675" ;
     5 ./2/ImageDatabase/w4028320/view0170.dcm
     6 000005AA: (0020,0037) [DS/62] "0.999905\-1.11616e-10\-0.0137944\0.
000295905\0.99977\0.021449 " ;
     7 ./2/ImageDatabase/w4028320/view0210.dcm
     8 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
     9 ./2/ImageDatabase/w4028320/view0211.dcm
    10 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    11 ./2/ImageDatabase/w4028320/view0212.dcm
    12 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    13 ./2/ImageDatabase/w4028320/view0213.dcm
    14 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    15 ./2/ImageDatabase/w4028320/view0214.dcm
    16 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    17 ./2/ImageDatabase/w4028320/view0215.dcm
    18 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    19 ./2/ImageDatabase/w4028320/view0216.dcm
    20 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    21 ./2/ImageDatabase/w4028320/view0217.dcm
    22 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    23 ./2/ImageDatabase/w4028320/view0218.dcm
    24 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    25 ./2/ImageDatabase/w4028320/view0219.dcm
    26 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    27 ./2/ImageDatabase/w4028320/view0220.dcm
    28 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    29 ./2/ImageDatabase/w4028320/view0221.dcm
    30 000005A8: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    31 ./2/ImageDatabase/w4028320/view0222.dcm
    32 000005A8: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    33 ./2/ImageDatabase/w4028320/view0223.dcm
    34 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    35 ./2/ImageDatabase/w4028320/view0224.dcm
    36 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    37 ./2/ImageDatabase/w4028320/view0225.dcm
    38 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    39 ./2/ImageDatabase/w4028320/view0226.dcm
    40 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    41 ./2/ImageDatabase/w4028320/view0227.dcm
    42 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    43 ./2/ImageDatabase/w4028320/view0228.dcm
    44 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    45 ./2/ImageDatabase/w4028320/view0229.dcm
    46 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    47 ./2/ImageDatabase/w4028320/view0230.dcm
    48 000005AA: (0020,0037) [DS/60] "0.996306\0.00783452\0.0855103\0.
00827955\0.982424\-0.186478 " ;
    49 ./2/ImageDatabase/w4089355/view0007.dcm
    50 0000062A: (0020,0037) [DS/30] "1\-0\-0\0\-0.052336\-0.998629 "
;
    51 ./2/ImageDatabase/w4089355/view0008.dcm
    52 0000062A: (0020,0037) [DS/30] "1\-0\-0\0\-0.052336\-0.998629 "
;
    53 ./2/ImageDatabase/w4089355/view0009.dcm
    54 0000062A: (0020,0037) [DS/30] "1\-0\-0\0\-0.052336\-0.998629 "
;
    55 ./2/ImageDatabase/w4089355/view0010.dcm
    56 0000062E: (0020,0037) [DS/30] "1\-0\-0\0\-0.052336\-0.998629 "
; ;
-- portion of input -- end --

Relevant Pages