Re: Mirroring 2 Machines with rsync

From: Baby Peanut (baby_p_nut2_at_yahoo.com)
Date: 06/24/03 

Date: 24 Jun 2003 08:38:35 -0700

bazzz777@yahoo.com (bazzz777) wrote in message news:<6c9ad26f.0306230837.17005e63@posting.google.com>...
> Anyone care to comment on the following?
>
> I have 2 rp2400 machines each with an Infotrend 500G array
> and running HP-UX 11.11. I want one to be
> a redundant mirrored copy of the other. Every night I sync them
> up with the rsync utility, http://rsync.samba.org.
>
> So far, I exclude the following files:
>
> /etc/rc.config.d/netconf
> /etc/utmp
> /etc/opt/resmon
> /dev
>
> Everything seems OK but rsync spits out a couple of
> hundred unexplained errors every night.
>
> Anyone care to comment on this method of mirroring?
> Are there more files I should be excluding?
> Thanks for any observations, hints, suggestions, etc..

I'm currently mirroring an L box to a K box. More than 5TB of data is
moved but runs are only done twice a week since they take so long.

The rsync utility has one big problem with scaling up: tables in core
(RAM) memory that grow to huge sizes when there are huge numbers of
files. I work around it by breaking up what rsync pushes into pieces
that are pushed via separate invocations of rsync.

I also push the rsync through OpenSSH to get compression. If you are
syncing compressed stuff this may not be useful to you.

Here is a shell script fragment that I modified to conceal the true
nature of the real shell script that is run.

dosync() {
   if [ ! -d $1 ]; then
     echo \\n$1 is not a directory, skipping.
     continue
   fi

   rsync -a --delete --stats --exclude=unwanted-patteren\* \
    --rsh="/usr/local/bin/ssh -C" \
    $i/ target-computer:$i \
>> /var/log/rsync-log-file 2>&1
}

dosync /usr
dosync /home
etc.

Relevant Pages

  • Re: Mirroring 2 Machines with rsync
    ... >> Anyone care to comment on this method of mirroring? ... > I'm currently mirroring an L box to a K box. ... > The rsync utility has one big problem with scaling up: ... > dosync /usr ...
    (comp.sys.hp.hpux)
  • Problem setting up a mirror
    ... I've installed and configured the anonftpsync script as per its instructions, but when I run it the log says the following: ... rsync: connection unexpectedly closed ... mirroring /pool exited with exitcode 12 ... To UNSUBSCRIBE, email to debian-user-REQUEST@xxxxxxxxxxxxxxxx with a subject of "unsubscribe". ...
    (Debian-User)
  • Re: mirroring a disk remotely in linux; alternatives to rsync
    ... data at the remote end without unmounting it - this may make it ... mirroring or continuous mirroring of the files, ... I have nothing against rsync. ... Instead of such a userspace solution if I had a filesystem that could ...
    (comp.os.linux.misc)
  • Re: FIN_WAIT_[1,2] and LAST_ACK
    ... it pays attention to /robots.txt. ... But, I am writing my own -- I don't want to use rsync, wget, anything like ... >portions at the filesystem level rather than mirroring via HTTP requests. ...
    (freebsd-net)
  • Re: Fedora 4 ?
    ... >> I don't know for sure, but I'm pretty confident that it will be. ... Care ... > You already placed a wager on system integrity in contrast to waiting 24 ... by rsync and run against the ones you already have. ...
    (Fedora)