Multi-machine mirroring choices

With the introduction of zfs to FreeBSD 7.0, a door has opened for more
mirroring options so I would like to get some opinions on what direction
I should take for the following scenario.

Basically I have 2 machines that are "clones" of each other (master and
slave) wherein one will be serving up samba shares. Each server has one
disk to hold the OS (not mirrored) and then 3 disks, each of which will
be its own mountpoint and samba share. The idea is to create a mirror of
each of these disks on the slave machine so that in the event the master
goes down, the slave can pick up serving the samba shares (I am using
CARP as the samba server IP address).

My initial thought was to have the slave set up as an iscsi target and
then have the master connect to each drive, then create a gmirror or
zpool mirror using local_data1:iscsi_data1, local_data2:iscsi_data2, and
local_data3:iscsi_data3. After some feedback (P.French for example) it
would appear as though iscsi may not be the way to go for this as it
locks up when the target goes down and even though I may be able to
remove the target from the mirror, that process may fail as the "disk"
remains in "D" state.

So that leaves me with the following options:
1) ggated/ggatec + gmirror
2) ggated/ggatec + zfs (zpool mirror)
3) zfs send/recv incremental snapshots (ssh)

1) I have been using ggated/ggatec on a set of 6.2-REL boxes and find
that ggated tends to fail after some time leaving me rebuilding the
mirror periodically (and gmirror resilvering takes quite some time). Has
ggated/ggatec performance and stability improved in 7.0? This
combination does work, but it is high maintenance and automating it is a
bit painful (in terms of re-establishing the gmirror and rebuilding and
making sure the master machine is the one being read from).

2) Noting the issues with ggated/ggatec in (1), would a zpool be better
at rebuilding the mirror? I understand that it can better determine
which drive of the mirror is out of sync than can gmirror so a lot of
the "insert" "rebuild" manipulations used with gmirror would not be
needed here.

3) The send/recv feature of zfs was something I had not even considered
until very recently. My understanding is that this would work by a)
taking a snapshot of master_data1 b) zfs sending that snapshot to
slave_data1 c) via ssh on pipe, receiving that snapshot on slave_data1
and then d) doing incremental snapshots, sending, receiving as in
(a)(b)(c). How time/cpu intensive is the snapshot generation and just
how granular could this be done? I would imagine for systems with litle
traffic/changes this could be practical but what about systems that may
see a lot of files added, modified, deleted to the filesystem(s)?

I would be interested to hear anyone's experience with any (or all) of
these methods and caveats of each. I am leaning towards ggate(dc) +
zpool at the moment assuming that zfs can "smartly" rebuild the mirror
after the slave's ggated processes bug out.

Sven

Attachment: signature.asc
Description: This is a digitally signed message part

Relevant Pages

  • Sanity check of filesystem design
    ... I'm contemplating a major overhaul of the filesystems on my Ultra 20 and would like a sanity check before I start reconfiguring the drives. ... I'm already using ZFS and it seems pretty straight forward. ... boot environment slices for live upgrade ... ZFS pool mirror for home ...
    (comp.unix.solaris)
  • Re: Sanity check of filesystem design
    ... I'm already using ZFS and it seems ... boot environment slices for live upgrade ... ZFS pool mirror for home ... mirroring root is to simplify keeping the 2nd BE in sync w/ the 1st. ...
    (comp.unix.solaris)
  • Re: zfs on a SAN
    ... group of storage engineers who run our EMCs and brocade switches. ... existing technologies offer what ZFS does: ... hope it comes from the other half of the mirror. ... You can do that but it would cost you twice again the number of disks. ...
    (comp.unix.solaris)
  • Re: zfs - move pool to new device?
    ... mirror vdevs and expected reads to only occur from two drives at a ... time (single half of a mirror). ... Since ZFS is smart enough to know ... yes but all your drives are likely identical and on the same kind of ...
    (comp.unix.solaris)
  • Re: Can Disk mirroring/db snapshots replace the usage of taking database backups?
    ... In order to use ontape for a logical log restore following an external restore you have to have IDS 10.00 or later. ... OK, so I'm assuming that the array is keeping an up-to-date copy on another structure somewhere, like EMC Symmetrics, and the snapshot operates by breaking that remote mirror, copying the remote disk off somewhere else, then reestablishing the remote mirror. ... Keep in mind again that if you are not running IDS 11.10 or later you will not be able to roll forward any logical logs after a restore. ...
    (comp.databases.informix)