[Summary] saving disksuite info while rebuilding server

• Previous message: NKS: "VPN automounts - direct maps"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Date: Tue, 10 Feb 2004 14:57:18 -0500
To: Sun Managers <sunmanagers@sunmanagers.org>

Finally done. Got a number of replies, some of which were useful. None
fully covered the bases. I put in a support call to Sun and fleshed out
the details (got to speak with a Network Storage Services Technical
Support Engineer). Original message first. Then a summary of how I
accomplished what I needed to do.

---------------

Chris Hoogendyk

-
  O__ ---- Network Specialist & Unix Systems Administrator
 c/ /'_ --- Library Information Systems & Technology Services
(*) \(*) -- W.E.B. Du Bois Library
~~~~~~~~~~ - University of Massachusetts, Amherst

<choogend@library.umass.edu>

---------------

-------- Original Message --------
Subject: saving disksuite info while rebuilding server
Date: Fri, 30 Jan 2004 13:24:57 -0500
From: Chris Hoogendyk <choogend@library.umass.edu>
To: Sun Managers <sunmanagers@sunmanagers.org>

I'll try this once again.

I got one reply that had some useful information, but I need more guidance.

I have an E250 with an external D1000 all set up with DiskSuite,
including mirrored root drive and raid 10 on the D1000.

It is running Solaris 7.

I need to rebuild it (and repartitian the root drive) by installing
Solaris 8 without losing the state database information or any of the
data on the external D1000.

Can anyone give me assurrances, guidance, steps ....

Is it possible to install Solaris 8, repartitian the root drive, and
keep the rest of the system intact?

---------------

Chris Hoogendyk

-
   O__ ---- Network Specialist & Unix Systems Administrator
  c/ /'_ --- Library Information Systems & Technology Services
 (*) \(*) -- W.E.B. Du Bois Library
~~~~~~~~~~ - University of Massachusetts, Amherst

<choogend@library.umass.edu>

---------------

_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers

SUMMARY -- SUMMARY -- SUMMARY -- SUMMARY

I found
<http://docs.sun.com/db/doc/806-3206/6jccd0hcb?a=view> which
goes through the updgrade in some detail. However, I have a corrupted
system (not the metadb or anything like that). My Solaris 7 system was
hacked and I can't trust it. I need to wipe it, not upgrade it. The
documentation
<http://docs.sun.com/db/doc/806-3206/6jccd0hca?a=view> warns
"Caution - If a full install of Solaris is performed, all DiskSuite
configuration information will be lost. Losing the DiskSuite
configuration information can result in data loss or complete system
failure."

I really wanted to re partitian the root drive and start fresh.

The critical data is on external arrays. Those are the metadevices that
need to be brought up without loss after the rebuild. The documentation
doesn't tell how to do this, it seems to imply that you shouldn't. However,
Sun support helped me through it.

Step 1: make sure you have tape backups of everything.

I had an old workgroup storage array with six 18G drives formatted RAID5
that only had about 11G of data at the moment. This is the one I was
concerned about getting back up. So it was easy to do a tape of it just
in case.

My D1000 had 170G of data and until I could do the rebuild, my only
means of backing it up was DDS/3 tapes which would take several days and
upwards of a dozen tapes (the data is mostly tiff so it is already
compressed). Fortunately, I had configured the D1000 as RAID 10, so I
basically had two full copies on two separate controllers. Inefficient
storage utilization, but very fast and secure. That meant that I could
try to bring up one controller and then add the other as a mirror if it
worked.

My root stuff was going to get blown away anyway with a repartitian and
full install of Solaris 8. So, have at it. I did have tape backups
anyway so that I could pull individual files if I needed them.

Step 2: specific information needed for the DiskSuite rebuilding process.

# metastat -p > ~chris/backup/md.orig.tab
# metadb > ~chris/backup/metadb.orig
# df -k > ~chris/backup/df.orig
# cp /etc/vfstab ~chris/backup/vfstab.orig
# cp /etc/system ~chris/backup/system.orig

Find the directory that has md.cf and/or md.conf (could be
/etc/opt/SUNWmd) and copy this whole directory to ~chris/backup

Copy all these files to another server where you will have access to
them during the install and configuration. If necessary do it using a
tape and ufsdump/ufsrestore.

I had an advantage that the internal drives were all 9G, the old
workgroup array had all 18G drives, and the D1000 had all 36G drives. So
after the install, I could diagnose which was which using prtvtoc, just
in case they somehow got assigned differently during the full install.
Didn't happen. But, if this is a concern, you might want to also grab
copies of the /dev/dsk, /devices and /etc/path_to_inst for reference.

Step 3: boot off install CD and do a full install.

Scary. Just what the documentation I found warned against. Even if I had
a mirrored root drive, it wouldn't matter, because I am going to rebuild
all that. Booted off a CD, it won't see it that way anyway.

So, now I can specify custom partitianing, use a 9G for root, a 9G for
/usr/local, a 9G for /export/home, and a 9G left over, not touching the
external array drives. Specified /var as a separate partitian of / just
so I could have it big enough but also not over-run everything else if
it got out of hand.

Be sure to select DiskSuite 4.2.1 as an install option.

Take care of securing the system. Shut down un-needed ports in
inetd.conf, pull down latest recommended & security patches, do
tcp_wrappers, etc., etc., ....

Step 4: OK, 3 all done, rebooted, now configure the arrays.

First, there are no state databases. The first one needs to be forced. I
created them exactly as they were according to my metadb.orig. Turns out
this was a problem, not because the strategy was necessarily wrong, but
because the configuration of my old workgroup array was wrong originally
when it was set up almost 5 years ago before I had any training. It had
been set up with the state databases and the actual data on the same
partitian. The drives come from Sun with s0 and s1 small, s2 everything,
and s6 what's left after s0 and s1. This was set up with an s7 also
everything and the state database and data both on s7.

Anyway, based the metadb.orig

# metadb -a -f c1t1d0s7 c1t2d0s7 c1t3d0s7 c1t4d0s7

This actually ended up blowing the data on that array. Sort of expected.
I later reconfigured it the way it should have been and recovered it
from tape.

Then, based on the md.orig.tab, and doublechecking with prtvtoc to see
if it was the right drives:

# prtvtoc /dev/rdsk/c3t0d0s2

    <output snipped>

# metainit d21 1 6 c3t0d0s6 c3t1d0s6 c3t2d0s6 c3t3d0s6 c3t4d0s6 c3t5d0s6
-i 64b

# metainit d20 -m d21

# metainit d22 1 6 c2t0d0s6 c2t1d0s6 c2t2d0s6 c2t3d0s6 c2t4d0s6 c2t5d0s6
-i 64b

# metattach d20 d22

Then, based on the vfstab.orig, I edited /etc/vfstab and added the two
lines for the external arrays. Then create the mount point corresponding
to these and mount the D1000.

# mkdir /array1

# mkdir /array2

# mount /array2

Then "cd /array2" and check it out. Everything was there! Cool. That was
the important one.

Then I had to rebuild the old workgroup array from scratch. But first, I
needed to move the state databases off of it. later I could put them
back on. Need to maintain some at all times (not delete them all and
then add some somewhere else).

# metadb -a c3t0d0s1 c3t1d0s1 c3t2d0s1 c2t0d0s1 c2t1d0s1 c2t2d0s1

Note that the D1000 had been set up with the data on s6 and the state
databases on s1, so they are separate partitians and utilize the default
Sun formatting that they came with.

Now,

# metadb -d c1t1d0s7 c1t2d0s7 c1t3d0s7 c1t4d0s7

And I'm ready to rebuild this array from scratch.

# metainit d10 -r c1t6d0s6 c1t5d0s6 c1t4d0s6 c1t3d0s6 c1t2d0s6 c1t1d0s6
-i 32b

Wait a long time.

# newfs /dev/md/rdsk/d10

Wait a long time.

# fsck /dev/md/rdsk/d10

# mount /array1

# cd /array1

# ufsrestore -if /dev/rmt/0n

Recover all the data. (Wait a long time.) Check to see that it is there
and alright.

Finally,

# metadb -a c1t1d0s1 c1t2d0s1 c1t3d0s1

This means I have 3 state databases on each external controller (the
D1000 has two), and should still have a majority if one goes down. If
the D1000 went down entirely, I would have only 3 of 9, but I could
force it from there if need be. However, at that point I would be more
concerned with getting the D1000 back up. If the old array went down,
the D1000 would still be up and would have a majority.

Then check everything with metadb, metastat, etc.

Finally, make sure user accounts are all set, users have access, Samba
is set up properly, etc.

It was a lot of trauma, but in the end it was not really that difficult.
If I have to do it again, I could probably do it in a day or so. Also,
since I have the old workgroup array configured properly (data on s6 and
state database on s1), I would be able to bring it back up without the
time consuming initialization, newfs and tape recovery. I would be able
to do

# metainit d10 -r c1t6d0s6 c1t5d0s6 c1t4d0s6 c1t3d0s6 c1t2d0s6 c1t1d0s6
-k -i 32b

Note the -k near the end. For a raid device, that tells metainit that
the device has already been raided and just needs to have its
configuration set up and confirmed.

I think that is everything. Now I can proceed with getting Netbackup
installed, configured and running.

---------------

Chris Hoogendyk

-
   O__ ---- Network Specialist & Unix Systems Administrator
  c/ /'_ --- Library Information Systems & Technology Services
 (*) \(*) -- W.E.B. Du Bois Library
~~~~~~~~~~ - University of Massachusetts, Amherst

<choogend@library.umass.edu>

---------------
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers

• Previous message: NKS: "VPN automounts - direct maps"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]