Dreadful gmirror performance, though each half works fine

I am trying to run a system with a pair of drives mirrored under
gmirror, one of thhem being local and the other remote using ggated.

I've measured the performance of both the local and the remote drive
individually, and they are both fine - can get about 70meg/second out
of them. I've also created mirrors with oone drive in of each of thhem, and
these perform fine too - i.e. each mirror with one drive still gives me
around 65-70 meg/second.

When I create a mirror by taking one of them and adding the other the mirror
resyncs at full speed (i.e. I see network traffic of around 70 meg/second
and the time taken to complete the resync correspnds to that rate).

...and then the performance of the drive drops to a few k/second ?! I am
running myself on top of the mirror drives, but it runs like treacle. During
this time there is no intensive CPU activity, no intensive network activits
and no intensive disc activity - the machine just seems completely cuiqre.
but doing an 'ls' on the mirror drive takes forever.

If i then take a drive out using 'gmirror deactivate' the command freezes
up for a couple of minutes, but enevtiually comes back. I then
do a 'gmirror status' and that also takes a minute or so to come back. During
the time these commands are running I get 100% disc activity in 'gstat' on
the drive I took out - though almost no actual data being trasferred.

...at this point I am back to the situation I started with (a mirror with
only one disc in it) but the performance is still horrible. I can stop
the mirror, detach the ggate drive, reattach, restart - still horrible
performance, even though I have now gone back to the situation I
had before.

Has anyone else seen this ? I've tried various configurations, accross
several machines, all with the same result. It looks like ggate gets itself
into a state where it is suddenly very slow, and then cannot recover from
that. This doesnt happen with local gmirrors, nor if I replace the
underlying ggate with iscsi (though that causes kernel panics but that's
another story).

Any advice ?

-pete.

PS: All machines are amd64/7.0-RELEASE


_______________________________________________
freebsd-stable@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: geom - help ...
    ... gstripe st1 da1 da2 ... gmirror drive st1 st2 ... easier and faster--you only need to rebuild component mirror (which involves ... I have three drives (two of which were scavenged from other ...
    (freebsd-questions)
  • Re: RAID1 mirroring w/ 5.3 choices?
    ... particularly how gmirror and atacontrol relate to each other? ... fdisk-partition and bsdlabel both drives ... resynching to the other half of the mirror pair. ...
    (comp.unix.bsd.freebsd.misc)
  • Re: Mirroring: gvinum or gmirror?
    ... I want to set the drives up as a mirror for data redundancy. ... seem to suggest that gvinum hasn't completely stabilised for production yet. ... I run gmirror on all machines which don't have a hardware RAID ...
    (freebsd-questions)
  • Re: Worst monday of my life - please help...
    ... If that's the case, restoring the SQL ... but it finally did install with a few hiccups. ... >> or just leave the mirrored drives in the server on the secondary IDE ... I decided it was time to re-add the mirror. ...
    (microsoft.public.windows.server.sbs)
  • Re: Realistic drive arrays
    ... > may be an opportunity to swap 2 of the disks for a pair of 143Gb drives ... >> two drives in the same mirror failing within the parts replacement SLA of ... >> How much memory the store.exe process uses really depends on how much ... >>> copy of the database on the disk. ...
    (microsoft.public.exchange.setup)