Re: Options for synchronising filesystems

From: Isaac Levy (ike_at_lesmuug.org)
Date: 09/26/05

  • Next message: Eric Anderson: "Re: Options for synchronising filesystems" 
    Date: Mon, 26 Sep 2005 14:16:31 -0400
To: Brian Candler <B.Candler@pobox.com>

    Hi Brian, All,

    This email has one theme: GEOM! :)

    On Sep 24, 2005, at 10:10 AM, Brian Candler wrote:

    > Hello,
    >
    > I was wondering if anyone would care to share their experiences in
    > synchronising filesystems across a number of nodes in a cluster. I
    > can think
    > of a number of options, but before changing what I'm doing at the
    > moment I'd
    > like to see if anyone has good experiences with any of the others.
    >
    > The application: a clustered webserver. The users' CGIs run in a
    > chroot
    > environment, and these clearly need to be identical (otherwise a
    > CGI running
    > on one box would behave differently when running on a different box).
    > Ultimately I'd like to synchronise the host OS on each server too.
    >
    > Note that this is a single-master, multiple-slave type of filesystem
    > synchronisation I'm interested in.

    I just wanted to throw out some quick thoughts on a totally different
    approach which nobody has really explored in this thread, solutions
    which are production level software. (Sorry if I'm repeating things
    or giving out info yall' already know:) 

    --
Geom:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/geom- 
intro.html
The core Disk IO framework for FreeBSD, as of 5.x, led by PHK:
http://www.bsdcan.org/2004/papers/geom.pdf
This framework itself is not as useful to you as the utilities which  
make use of it,
--
Geom Gate:
http://kerneltrap.org/news/freebsd?from=20
Network device-level client/server disk mapping tool.
(VERY IMPORTANT COMPONENT, it's reportedly faster, and more stable  
than NFS has ever been- so people have immediately and happily  
deployed it in production systems!)
--
Gvinum and Gmirror:
Gmirror
http://people.freebsd.org/~rse/mirror/
http://www.ie.freebsd.org/doc/en_US.ISO8859-1/books/handbook/geom.html
(Sidenote: even Greg Lehey (original author of Vinum), has stated  
that it's better to use Geom-based tools than Vinum for the  
forseeable future.)
--
In a nutshell, to address your needs, let me toss out the following  
example setup:
I know of one web-shop in Canada, which is running 2 machines for  
every virtual cluster, in the following configuration:
2 servers,
4 SATA drives per box,
quad copper/ethernet gigabit nic on each box
each drive is mirrored using gmirror, over each of the gigabit  
ethernet nics
each box is running Vinum Raid5 across the 4  mirrored drives
The drives are then sliced appropriately, and server resources are  
distributed across the boxes- with various slices mounted on each box.
The folks I speak of simply have a suite of failover shell scripts  
prepared, in the event of a machine experiencing total hardware failure.
Pretty tough stuff, very high-performance, and CHEAP.
--
With that, I'm working towards similar setups, oriented around  
redundant jailed systems, with an eventual end to tie CARP (from pf)  
into the mix to make for nearly-instantaneous jailed failover  
redundancy- (but it's going to be some time before I have what I want  
worked out for production on my own).
Regardless, it's worth tapping into the GEOM dialogues, as there are  
many new ways of working with disks coming into existence- and the  
GEOM framework itself provides an EXTREMELY solid base to bring  
'exotic' disk configurations up to production level quickly.
(Also noteworthy, there's a couple of encrypted disk systems based on  
GEOM emerging now too...)
--
Hope all that helps,
Best,
.ike
_______________________________________________
freebsd-isp@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-isp
To unsubscribe, send any mail to "freebsd-isp-unsubscribe@freebsd.org"
  • Next message: Eric Anderson: "Re: Options for synchronising filesystems"

    Relevant Pages