bugged sysinstall, bsdlabel, zfs, gmirror - recept for disaster :)
- From: Bartosz Stec <admin@xxxxxxx>
- Date: Wed, 03 Sep 2008 14:28:38 +0200
Hello there!
Here's my story, hopefully some of you won't follow my steps and avoid some troubles :)
Yesterday I've decided that's about time to test zfs functionality on my home server PC (i386 FreeBSD 7.1-pre) . A couple of weeks ago I bought new desktop PC (with SATA), so I had a bunch of PATA disks from old one to use in server. Lucky me - there was 3 HDD at size 40GB - RAIDZ was on approch!
So after a thirty minutes I had a plan, and my server had 4 disks connected - one 20GB with actual system (ad1), and three 40GB to replace actual system (ad[023]).
Plan was simple:
1. csup freebsd-stable
2. follow the tuning guide for zfs, rebuild world, kernel, and
follow system upgrade
3. Reboot in single user mode
4. fdisk new disks with sysinstall using one big slice for every disk
5. bsdlabel every new disk with sysinstall using: 1GB for /, 512MB
for swap, and rest unused (for ZFS)
6. gmirror -n -v -b round-robin boot ad0s1a ad2s1a ad3s1a
7. newfs /dev/mirror/boot
8. mount /dev/mirror/boot /mnt && cd /mnt
9. dump -h 0 -L -f - -C 32 / | restore rf -
10. zpool create tank raidz ad0s1d ad2s1d ad3s1d
11. zfs create new cool filesystems :)
12. dump | restore old ufs2 filesystem to new cool zfs filesystems :)
13. changing mount points from tank/foo to /foo
14. edit new fstab on mirror by replacing root mount point by "boot"
mirror, adding new swaps and remove ald ones and all fs now placed
on zpool
15. power off system, detach ad1 and power on new system in mixed
gmirror - raidz environment. Yay!
Well...it has almost works. Sysinstall screw it up. I was always too lazy to read man bsdlabel, that's why I've been using this "nice" tool for disk related tasks. Such a mistake!
Problem with labels created with sysinstal, is that it aks for a mount point for every partition in slice. Well, in my case it was unwanted behaviour, so on every disk I created first:
a: / b: swap
c: none
d: /foo
Then by using "M" key I removed mount points and saved changes with "W". At this point everything seems ok. So I've added gmirror to loader.conf and run "gmirror label -n -v -b round-robin boot ad0s1a ad2s1a ad3s1a". Still ok until now. Next step - kldload geom_mirror. Here's disaster! System became unresponsible and hangs after a while. Reboot didn't help, just after gmirror module was loaded by kernel, screen was flooded with messages:
WARNING: Expected rawoffset 0, found 63
andy didn't boot. I've made system start only because an old drive ad1 has no gmirror module added to loader.conf. So after reboot I've cleared metadata on providers and made some another attempts, but results were always the same. Finally I have found explanation for this issue. Man bsdlabel says:
/offset/ The offset of the start of the partition from the beginning of
the drive in sectors, or *** to have *bsdlabel* calculate the correct
offset to use (the end of the previous partition plus one, ignor-
ing partition `c'). For partition `c', *** will be interpreted as
an offset of 0. The first partition should start at offset 16,
because the first 16 sectors are reserved for metadata.
So proper labels for disks should be (and they are now):
# /dev/ad0s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 2097152 16 4.2BSD 0 0 0
b: 1048576 2097168 swap
c: 78156162 0 unused 0 0 # "raw"
part, don't edit
d: 75010418 3145744 unused 0 0
Problem was - Sysinstall has placed partition "a:" starting with offset 0! This is what happens when you don't RTFM :) I assume that this bug occured because I created mount point for root on ad[023]s1a and removed it after, than saved label. It seems that GEOM framework didn't expect this, neither maual for bsdlabel. I think that should be fixed somehow.
Fortunately manually editing labels by "bsdlabel -e" wasn't so hard as I expected. This is how I made everything back to normal:
a: 1024M * 4.2BSD 0 0 0
b: 512M * swap
c: 78156162 0 unused 0 0 # "raw"
part, don't edit
d: * * unused 0 0
After that, gmirror has stopped pissing me off, and I finished my plan, as below:
# zpool status
pool: tank
state: ONLINE
scrub: scrub completed with 0 errors on Wed Sep 3 10:10:07 2008
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1 ONLINE 0 0 0
ad0s1d ONLINE 0 0 0
ad2s1d ONLINE 0 0 0
ad3s1d ONLINE 0 0 0
errors: No known data errors
# gmirror status
Name Status Components
mirror/boot COMPLETE ad0s1a
ad2s1a
ad3s1a
Good luck with ZFS everyone! :) And RTFM ;)
--
Bartosz Stec - specjalista ds. IT
AUXILIA Spółka z o.o.
_______________________________________________
freebsd-stable@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@xxxxxxxxxxx"
- Follow-Ups:
- Prev by Date: Re: vfs.ffs.rawreadahead
- Next by Date: Re: vfs.ffs.rawreadahead
- Previous by thread: vfs.ffs.rawreadahead
- Next by thread: Re: bugged sysinstall, bsdlabel, zfs, gmirror - recept for disaster :)
- Index(es):
Relevant Pages
|
