Re: Backups: rsync, software RAID, other strategies?
From: Bob Johnson (bob89_at_bobj.org)
Date: 03/12/04
- Previous message: Website Toolbox: "help configuring network"
- In reply to: Dany Nativel: "Re: Backups: rsync, software RAID, other strategies?"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
To: Dany Nativel <dany_list@natzo.com> Date: Fri, 12 Mar 2004 00:14:51 -0500
Thanks for pointing me to these tools. I may not use Unison on this
particular project, but I think it will be the next project after the
backup server is finalized!
- Bob
On Monday 08 March 2004 02:22 am, Dany Nativel <Dany Nativel
<dany_list@natzo.com>> wrote:
> Hi Bob,
>
> I use the following configuration on my file server.
> It's a small cube based on a low power mini-ITX EPIA 5000 motherboard
> (fanless) and running two 120GB HDDs.
> I looked at RAID but it doesn't help solving one of the potential
> issue ... me, user removing files that are not supposed to be removed
> so incremental backup is a plus. This baby is powered by FreeBSD
> 5.2.1 (used to be Debian).
>
> DISK 0 (live)
> 128M ad0s1a /
> 512M ad0s1b swap
> 128M ad0s1d /var
> 200M ad0s1e /tmp
> 3200MB ad0s1f /usr
> 110GB ad0s2d /data
>
> DISK 1 (Backup)
> ad2s1d 128M /backup/os/root
> ad2s0b 512M swap
> ad2s1e 128M /backup/os/var
> ad2s1f 200M /backup/os/tmp
> ad2s1g 3199M /backup/os/usr
> ad2s2d 108G /backup/data/backup
>
>
> I use 3 different programs :
> - Unison (http://www.cis.upenn.edu/~bcpierce/unison/): 2-way
> synchronization using rsync/ssh, multi-platform graphical interface.
> I can have the same files on my file server, laptop running win2k as
> well as my desktop running BSD. Very convenient especially with
> laptops when you can't be connected all the time.Very fast too (only
> transmit diffs) - rsync (man rsync) : typical rsync that will mirror
> the source to the destination
> - rdiff-backup (http://rdiff-backup.stanford.edu/index.html): it's
> based on rsync but you get the advantage of incremental backups so
> you can restore from a specific date. You can also purge the backup
> by removing old stuff. No fancy file format, just .gz for the
> diffs.
>
> Here is how I use those tools :
> /data/current/user0_live (DISK 0) <----> UNISON : 2-way
> synchronization with laptop/desktop
> /data/current/user0_incremental (DISK 0) <----> RDIFF-BACKUP :
> incremental backup of user0_live using RDIFF system
> /backup/data/backup/user0_incremental (DISK1) <----> RSYNC : quick
> mirror of the already incremental backup
>
> /backup/data/backup/pictures (DISK1) <----> RDIFF-BACKUP
> : in this case, rdiff-backup between drive0 and drive1 (no
> incremental on disk0)
>
> /backup/os/root (DISK1) <----> DUMP : 1:1 copy of the live root
> fs /backup/os/tmp (DISK1)
> /backup/os/var (DISK1)
>
> /backup/os/usr (DISK1) <----> RSYNC : (with -delete option)
> for a quick mirror of current /usr
>
> PS: for user0, there are two copies of the data on disk0, 1 live
> synchronized with Unison and another one which is an incremental of
> the first one. For less critical data (like /data/current/pictures)
> I use rdiff-backup between disk0 and disk1. In that case I would lose
> incremental backups if disk1 fails.
>
> I've simulated a crisis situation by removing the drive0 and swapping
> it with drive1. It worked (except for those entries in fstab
> referring to disk1).
>
> I have a cron job taking of the different backups at night.
>
> #!/bin/sh
>
> # Duplicate /
> # erase slive before mirroring, any other way?
> umount /backup/os/root
> newfs /dev/ad2s1a
> mount /backup/os/root
> # dump with 'live filesystem' option
> dump -0 -L -f - /dev/ad0s1a | (cd /backup/os/root ; restore -r -v -f
> -)
>
> # Duplicate /var
> umount /backup/os/var
> newfs /dev/ad2s1d
> mount /backup/os/var
> dump -0 -L -f - /dev/ad0s1d | (cd /backup/os/var ; restore -r -v -f
> -)
>
> # Duplicate /tmp (probably a wate of time)
> umount /backup/os/tmp
> newfs /dev/ad2s1e
> mount /backup/os/tmp
> dump -0 -L -f - /dev/ad0s1e | (cd /backup/os/tmp ; restore -r -v -f
> -)
>
> # incremental backup of the ./pictures directory on the second drive
> rdiff-backup /data/current/pictures /backup/data/backup/pictures
>
> # First, incremental of the user0_live dir on the same drive then
> rsync on the second drive
> rdiff-backup /data/current/user0_live /data/current/user0_incremental
> rsync -a --delete /data/current/user0_incremental/
> /backup/data/backup/user0_incremental
>
>
> The only I don't like is the NEWFS command. Is there a cleaner way to
> do this dump ?
>
> I use this configuration is a non-critical installation (my house)
> but it has been serving its purpose so far.
> Dany
>
>
> PS: On the rdiff-backup webpage there is a link to another tool call
> duplicity (http://rdiff-backup.stanford.edu/duplicity.html). You can
> do remote backup but in that case the image can be stored on a remote
> FTP server and encrypted with GPG... sweet if you're planning to use
> your ISP's disk space for backups!
>
> Bob Johnson wrote:
> >A bunch of related questions:
> >
> >I'm setting up a small mail and file server. The mail server part
> > will be Courier, while the file server part will primarily be used
> > via NFS and Samba to store backups of my desktop and laptop
> > computers.
> >
> >The system has a pair of WD1600JB 160 GB ATA 100 drives in it, both
> > on a single Promise PDC20268 UDMA100 controller, but each on a
> > separate channel (i.e. both are masters with no slaves). My plan
> > is to use one of the drives as a backup for the other. I want to
> > use a backup method that creates a mirror of the working drive so
> > that if it fails, I can simply mount the backup in place of the
> > working drive, and get back in business. The operating system will
> > (probably) not be on either of these drives, they will only host
> > /home where mail and backup files will reside.
> >
> >I've tentatively decided to synchronize the mirror to the working
> > drive with rsync run once a night (perhaps more often later). This
> > risks losing up to a day of mail, but that's probably not the end
> > of the world. The reasoning I used was that if I use software
> > RAID, an unexpected power failure during a large write operation
> > (yes, it will be on a UPS) could corrupt both drives. Running
> > rsync once a night would reduce the risk of a failure that damaged
> > both drives, since their write activity would not be so strongly
> > correlated.
> >
> >Is my fear of losing both drives in a software RAID (mirrored
> > drives) configuration a reasonable one? Or is that not going to
> > happen?
> >
> >If I use rsync with -delete to maintain a mirror of the primary
> > drive, what happens when the primary drive fails? Is there a
> > scenario that causes rsync to duplicate all the missing data on the
> > primary drive by deleting it from the mirror drive (I've heard of
> > this happening to someone, but I believe he was using a homegrown
> > perl script rather than rsync).
> >
> >Is Courier compatible with this scheme? Or does it care about inode
> >numbers or some such thing that will make the backup copy useless?
> >
> >Is there any chance it would make sense to use the Coda file system
> > for this?
> >
> >Do any of the answers change if the mail server ends up on a remote
> >system, but I still want the maildirs backed up on the file server?
> >
> >Any other suggestions that don't involve buying new hardware? An
> > IDE RAID controller would be nice, but buying one isn't on my list
> > of things to do. But if I DID break down and buy a new controller
> > card, what should it be?
> >
> >Thanks,
> >
> >- Bob
> >_______________________________________________
> >freebsd-questions@freebsd.org mailing list
> >http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> >To unsubscribe, send any mail to
> > "freebsd-questions-unsubscribe@freebsd.org"
_______________________________________________
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org"
- Previous message: Website Toolbox: "help configuring network"
- In reply to: Dany Nativel: "Re: Backups: rsync, software RAID, other strategies?"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
|
