SUMMARY: metadevice naming scheme

• Previous message: vishwanath.basavaiah_at_wipro.com: "Netmasks in Solaris"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Date: Fri, 29 Oct 2004 16:51:42 -0400
To: <sunmanagers@sunmanagers.org>

SUMMARY: What metadevice naming scheme do you recommend/use? (Original post at
bottom)

Thanks for taking the time to provide such great answers! Thanks: David,
Michael-John, Jeremy, Clive, Oliver, Paul, Thomas, Sonny, and Dan.

SHORT SUMMARY: SVM naming limitations stink. Base your metadevice names on
slices. Base your submirror names on controller, target, and slice numbers.
Consider using different naming schemes for the internally mirrored OS drive
versus external disk arrays. More detailed summary below.

Ironically, I attended a Sun Solaris 10 technical course this morning where
they covered ZFS. ZFS makes all of these petty meta-concerns a thing of the
past (or so they say). It's like SVM, UFS, VxVM, and VxFS all smashed together
into one super filing system. If you are at all interested in file system
management, see http://www.sun.com/2004-0914/feature/ It's currently
scheduled as a Solaris 10 enhancement. (*cough* unproven future-ware *cough*)

DETAILED SUMMARY FOLLOWS...
Most agreed metadevice naming is clunky due to SVM naming restrictions. Nobody
claimed to have the one right answer; instead, a common suggestion was to use
combination of naming schemes depending on the storage layout.

Assuming a host with 2 internal drives for a mirrored OS, most people still
prefer to use the slice number as part of the mirror and submirror names. For
example:

  d0 = / on c1t0d0s0 / c1t1d0s0 (submirrors d10 & d20)
  d3 = /var on c1t0d0s3 / c1t1d0s3 (submirrors d13 & d23)
  d4 = /opt on c1t0d0s4 / c1t1d0s4 (submirrors d14 & d24)

This way, you know d13 is the "first" submirror of metadevice 3 which is based
on slice 3.

A similar approach was to base names on slice numbers, but also include the
controller number. Assuming the host has multiple controllers, this is a good
way to ensure controller diversity to For example:

  d10 = / on c1t0d0s0 / c1t1d0s0 (submirrors d100 & d110)
  d13 = /var on c1t0d0s3 / c1t1d0s3 (submirrors d103 & d113)
  d14 = /opt on c1t0d0s4 / c1t1d0s4 (submirrors d104 & d114)

Or, if hardware permits, use a separate controller:

  d10 = / on c1t0d0s0 / c2t0d0s0 (submirrors d100 & d210)
  d13 = /var on c1t0d0s3 / c2t0d0s3 (submirrors d103 & d213)
  d14 = /opt on c1t0d0s4 / c2t0d0s4 (submirrors d104 & d214)

After studying this approach a little further, it seems you could keep
metadevice names of d0, d3, and d4, but use the three digit submirror names to
capture controller numbers. Of course, this would only be useful on systems
with more than one disk controller. Dan also notes that using 3 digits may
require changes to your /kernel/drv/md.conf file:

"...The most recently evolved naming scheme uses mirror names in the d10-d99
range (i.e., two digits), with submirrors of three digits such as d101, d202,
etc. This normally means changing "nmd" in /kernel/drv/md.conf to a larger
number than the default 128. Usually I use 400. This requires a reconfigure
reboot.
..."

The above schemes are common and probably the easiest for an incoming admin to
understand at a glance. However, a readability and sorting issue I've always
complained about is the inability to have a metadevice named d00 or d01.
Thomas works around this using a small rotary shift:

".../dev/md/dsk/dXY represents the submirrors. Y is 0 or 1 to represent which
of the two disks (usually something like c1tYd0sZ), and X represents the
slice number + 1 (e.g. Z+1 = X); the plus one is required because metadisk
does not like stuff like d01 and d00. /dev/md/dsk/dX represents the mirrored
metadevice which gets mounted, consisting of submirrors dX0 and dX1, and X is
one more than the slice number on the raw devices the submirrors are made from
(c1t0d0sZ and c1t1d0sZ where Z+1=X)

Again, the +1 arises since d00 and d01 are problematic, and I figure the +1
across the board is better than having d0 have submirrors d10 and d11.
..."

Managing disk arrays is where things get really interesting. Michael-John
writes:

"...Say you've got a thumping big disk array with 22 disks, c1t0d0 through to
c1t21d0. Here's a naming scheme for putting all the disks in a mirrored soft
partition, then creating devices out of the big mirror:

metainit d9 - soft partition mirror
metainit d19 1 11 c1t0d0s0 c1t1d0s0 c1t2d0s0 c1t3d0s0, etc
metainit d29 1 11 c1t12d0s0 c1t13d0s0 c1t14d0s0, etc

Now create metadevices out of the soft partition:

metainit d39 -p d9 5g
metainit d49 -p d9 10g

So the naming scheme is - soft partition '9', and metadevices using this sp to
have a '9' somewhere in there. Or you could do sp=d100 and partitions: d100,
d101, d102, etc.
..."

It seems this naming scheme would work well on a host that had it's two
internal drives mirrored. Since you wouldn't have a slice 9 on the host OS
disk, you would immediately know any metadevices ending in a 9 were not slice
9; rather, they were part of the external disk array mirror. True, it doesn't
scale well if you wanted many more metadevices on the external array, but the
idea is you wouldn't need more metadevices, you would create more soft
partitions.

SUMMARY: Do you have a usual text file where you store meta-related info?

Everyone suggests adding comments to the /etc/lvm/md.tab file. Thanks to Paul
for a good example:

"...I put that sort of information in the SVM file for it. Namely,
/etc/lvm/md.tab. If you put the info there it gets backed up and you can add
comments to the file to provide any further details of the config. It also
frees you from having to specify parameters to init commands on the command
line. As an example, here's the tail end of one of my md.tab files (I leave
the comments at the beginning of the file in place as a quick reference on the
config :)):

mddb00 /dev/dsk/c0t0d0s4 \
        /dev/dsk/c0t8d0s4 \
        /dev/dsk/c0t9d0s4 \
        /dev/dsk/c0t10d0s4
#
# /
#
d0 -m d10
d10 1 1 /dev/dsk/c0t0d0s0
d20 1 1 /dev/dsk/c0t8d0s0
#
# swap
#
d1 -m d11
d11 1 1 /dev/dsk/c0t0d0s1
d21 1 1 /dev/dsk/c0t8d0s1
#
# /usr
#
d5 -m d15
d15 1 1 /dev/dsk/c0t0d0s5
d25 1 1 /dev/dsk/c0t8d0s5
#
# /var
#
d6 -m d16
d16 1 1 /dev/dsk/c0t0d0s6
d26 1 1 /dev/dsk/c0t8d0s6
#
# /tmp
#
d7 -m d17
d17 1 1 /dev/dsk/c0t0d0s7
d27 1 1 /dev/dsk/c0t8d0s7
#
# /nsr
#
d37 -m d47
d47 1 1 /dev/dsk/c0t9d0s7
d57 1 1 /dev/dsk/c0t10d0s7

Once SVM has been installed I add those entries to the md.tab file and
setup is as simple as:

# metainit d20
# metainit d10
# metainit d0
# metattach d0 d20

and after the resync is complete I'm ready to go. The primary metadevices
don't list the mirror device in their definition (i.e., d37 -m d47 rather
than d37 -m d47 d57) because older versions of disk suite had problems
initializing the primary device if the mirror was specified on it so you
specify it standalone and manually attach the
device to be safe. It always made sense to me to put this info in md.tab since
that's where SVM looks for it and it gives documentation on the config in case
the machine needs to be rebuilt. I've had the unfortunate experince of having
to rebuild a machine that was using ODS (early incarnation of SVM) and the
admin hadn't put the info in md.tab. Not something I want to do again soon.
:)
..."

On a side note, I've often found myself defending the notion that slice 7 on
all disks should be carved out with a little space "just in case." Even if you
don't initially plan on locating a metadb on it. Dan confirms my approach:

"...I use slice 7 on all disks for metadb replicas, although I don't
necessarily put replicas on every drive. It's just useful to have it there if
you need it.
..."

Hope this SUMMARY helps!
-John Christian

-----------------------------------------------------------------------------

---
From: sunmanagers-bounces@sunmanagers.org on behalf of John Christian
Sent: Thu 10/28/2004 8:07 AM
To: sunmanagers@sunmanagers.org
Subject: metadevice naming scheme?
Sun Managers,
What metadevice naming scheme do you recommend/use?
Do you have a usual text file where you store meta-related info? (such as
/etc/README.metainfo)
Many of the systems I build will eventually be taken over by other
sys-admins.
My primary goal is easily recognized simplicity over extravagance. (Although
I'm interested in hearing about both!) I'd like for those who inherit my
systems to say "Well, maybe I would've done it differently, but this does
make
sense."
In the past, I typically used Sun Volume Manager (Solstice Disk Suite) to
mirror the OS disk and used VERITAS for all other volumes. With SVM "free"
(and dollars tight), I have more clients opting for a full SVM solution with
no VERITAS volume management. Seems the traditional method of basing
metadevice names on disk slice #'s does not scale well.
I'm sure this question has been answered before, but I'd like to hear the
current trends. TIA for any input!
-John C.
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers

• Previous message: vishwanath.basavaiah_at_wipro.com: "Netmasks in Solaris"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]