Re: Best practices for using gjournal with gmirror?



On Wed, Jan 10, 2007 at 11:21:01PM -0500, John Nielsen wrote:
I have a few questions for pjd (or anyone else) about using gjournal,
particularly when used with gmirror.

1) I'm running 6-STABLE and plan to test with gjournal6_20061030.patch (from
the mailing list; updated version of 20061024 that applies cleanly). Is
there a better/newer version for -STABLE that I should use instead?

There probably should be a newer version as there were some minor
changes after I committed the code to HEAD. I'll try to create a new
patch during the weekend.

2) When using gjournal and for a gmirror volume, does the journal need to be
mirrored as well to maintain redundancy? If so, when storing the journal on
the same physical disks as the mirror, is it better to mirror at the slice
level (journal and fs on different partitions in the same mirror) or at the
partition level (journal and fs each have their own mirror) or does it
matter?

The problem with mirroring each partition/slice separately is that when
you have a crash, on boot, gmirror will start to rebuild all partitions
at once, which may be problematic. On the other hand, when you mirror
each partition/slice separately, and some partitions weren't modified in
last few seconds before the crash, gmirror will not resync them on boot,
so not entire disk will be synchronized.

When you run gjournal on top of gmirror/graid3 there is no need for
resync after a crash, so bascially all cons against mirroring the whole
disks and against mirroring partitions are no longer true. Both
configurations will work the same. In that case I'd suggest mirroring
the whole disks, because when one of your disks dies, you may just
replace it and be down with it. If you mirror partitions separately, you
first have to create partitions and insert each of them into their
mirrors, which is more complex than simple 'gmirror insert foo newdisk'.

3) I remember reading where pjd said that gjournal plus gmirror or graid3
would eliminate the need to re-sync the array after a crash. While clearly
a design goal, is that actually the case with the version of the patch
mentioned above? If so, are any config changes needed or will it just
happen automagically?

No, you need to:

# gmirror configure -F <mirror_name>

4) In the same vein as 3)--does a gjournal volume need to be fsck'ed after a
crash? If not, will it just work (e.g. fsck -p sees that the filesystem is
clean) or does it need to be disabled somehow?

Gjournaled file system has to be fscked, but only to handle orphaned
files. Such fsck on multiterabyte provider takes seconds, not hours.

5) Finally, how dangerous is this code? I realize it's experimental and only
plan to use it with data that has recent backups, but how much should I
worry about it blowing up my system or corrupting my files?

I'm using it in production, my customer using it in production on large
number of FreeBSD servers and I also have heard already many success
stories, BUT I still consider the code to be experimental.

--
Pawel Jakub Dawidek http://www.wheel.pl
pjd@xxxxxxxxxxx http://www.FreeBSD.org
FreeBSD committer Am I Evil? Yes, I Am!

Attachment: pgpT0LatLWVhE.pgp
Description: PGP signature



Relevant Pages

  • Re: gmirror problem on 5.3-R i386
    ... > I'm having a problem with gmirror. ... > first disk to the mirror after booting off the second drive. ... > status is a degraded mirror and I cannot see any of the partitions on ...
    (freebsd-questions)
  • Hand on gmirror (Was: Re: gmirror bugs, how many?)
    ... > All I'm saying is that you first should create a mirror, ... > partitions on ad0 you'll overwrite gmirror metadata. ... sigesc::root jcmendes gmirror list ...
    (freebsd-hackers)
  • Best practices for using gjournal with gmirror?
    ... When using gjournal and for a gmirror volume, does the journal need to be ... the same physical disks as the mirror, is it better to mirror at the slice ... I remember reading where pjd said that gjournal plus gmirror or graid3 ... would eliminate the need to re-sync the array after a crash. ...
    (freebsd-hackers)
  • Re: gjournal: journaled slices vs. journaled partitions
    ... I didn't want to only put gmirror and have full drives rebuild ... I went with this setup anyway and I made some reset tests to see what ... provider is directly on the mirror provider for all mirrored filesystems ... the ill-advised journaled small partition issue been successfully overcome ...
    (freebsd-questions)
  • Re: gmirror disk fail questions...
    ... mirror/gm0 DEGRADED ad4 ... I am guessing that a failing disk is responsible for the data ... the mirror is marked clean ad there's no warnings about ... I believe the kernel hides individual labels for a gmirror volume. ...
    (freebsd-questions)