Re: merging directories

From: Stephen Baynes. (stephen.baynes_at_soton.sc.philips.REMOVETHIS.com)
Date: 01/16/04 

Date: Fri, 16 Jan 2004 12:57:26 -0000

"Martin Trautmann" <t-use@gmx.net> wrote in message news:slrnc0d36e.5sk.t-use@ID-685.user.dfncis.de...
>
> Hi all,
>
> do you know any scripts around in order to merge two directories? It
> should be significantly better than a 'mv' or 'gtar -d' or basic rsync:
>
> Two directories may contain differently named, binary identical files
> (if anyone is interested: e.g. a spidered website for two different
> years and a reorganisation in between)
>
> Thus a first step would be to identify duplicates, e.g. by an ls -l,
> sorting by size, comparing the contents (e.g. by comparing an md5
> checksum, a binary diff, a 'cmp').

Here is a short script that finds duplicates in a directory tree by cksum.

#!/bin/sh

find "$@" -type f -exec cksum '{}' ';' | sort | awk '$1==lsum && hdone!=$1 { print $1; print " ",lfile ; hdone=$1 };$1==lsum {
print " ",$0 };{ lsum = $1; lfile = $0 }'

It might get you started.

Relevant Pages

  • Re: merging directories
    ... > do you know any scripts around in order to merge two directories? ... > Two directories may contain differently named, binary identical files ... The python script is the cleaner, sh the simplerand perl the ...
    (comp.unix.shell)
  • Re: Gordian knot with File::Find
    ... > However, I'd prefer relative paths, but if I change the original find to ... I used find2perl to create scripts for me using: ... All three cases search the whole directory tree. ...
    (comp.lang.perl.modules)
  • Is this a proper use of LOFS - loopback virtual file system
    ... We have a filesystem mountpoint /opt/home/coop where we have all our scripts ... and naviagte the /opt/home/coop directory tree to put and get files. ... what if we use lofs a process logs in sees /opt/home/coop and then navigates ...
    (comp.unix.solaris)