SUMMARY: verify AdvFS domain - fileset is corrupted

Shaun.Racine_at_intier.com
Date: 11/17/04

  • Next message: Bugs: "SUMMARY: pg_nreaders going negative"
    Date: Wed, 17 Nov 2004 16:03:31 +0000
    To: tru64-unix-managers@ornl.gov
    
    

    Hi Managers,

    Problem resolved by almost unanimous response, using command;
          fixfdmn

    I could not find this on the boot CD, but I was able to mount the root
    partition from HSG and copy to /tmp within boot shell and umount and run
    from there.

    Many thanks to all responders ;

    BL Venkatesh
    Dr Thomas Blinn
    Thomas Sjolshagen
    Bob Harris
    Lawrie Smith
    Bryan LaVelle
    Paul Crittenden
    John L. Lanier
    Martin Roende Andersen
    Robert Collins
    Derek Haining

    Best regards,
    Shaun

    All responses follow ;

    Original question
    ===============
    Hi Managers,

    V5.1A pk 0001

    I am attempting to correct fileset corruption on a LSM / AdvFS root_domain.

    I have booted from the Unix CD, accessed the shell prompt.
    Started LSM (after a lot of tries!)
    Linked in AdvFS root_domain to /etc/fdmns
    Verify does not fix the problem.

    Where do I go from here?

    Much obliged,
    Shaun Racine

    Here is the output from verify;

    # /sbin/advfs/verify -f -d root_domain
    verify: error [13] posting event: sys.unix.fs.advfs.fset.backup.lock
    +++ Domain verification +++

    Domain Id 3d7b53bb.0009d2b0

    Checking disks ...

    Checking storage allocated on disk /dev/vol/rootdg/rootvol

    Checking mcell list ...

    Checking that all in-use mcells are attached to a file's metadata mcell
    chain...

    Checking tag directories ...

    +++ Fileset verification +++

      +++++ Fileset root +++++

    Checking frag file headers ...

    Loop found in list of free frags on page 1040 of the frags file.
    Frags metadata file is corrupted!

    Fileset is corrupted! Attempts to use this fileset may result in
    system crashes or user data loss!

    verify: error [13] posting event: sys.unix.fs.advfs.fset.backup.unlock
    #

    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          "Venkatesh, BL"
                          <venkatesh.bl@> To: <Shaun.Racine@intier.com>
                                                   cc:
                          05/11/2004 11:41 Subject: RE: verify AdvFS domain - fileset is corrupted
                                                                                                                                          
                                                                                                                                          

    did u try 'salvage' to recover the files?

    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          "Dr
                          Thomas.Blinn@" To: Shaun.Racine@intier.com
                          <tpb cc:
                                                   Subject: Re: verify AdvFS domain - fileset is corrupted
                          05/11/2004 12:47
                                                                                                                                          
                                                                                                                                          

    You probably will need to recreate the domain and fileset and
    restore it from backup. It would be interesting to know if
    the cause of the failure was hardware or software, since the
    binary error log is in the /var hierarchy which may be intact,
    and the kernel message log is there also, you might be able to
    find some useful data if you can get to those files. If your
    root disk is bad, restoring the file systems onto it might be
    a bad idea.

    Tom

    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          "Sjolshagen,
                          Thomas (Tru64 To: <Shaun.Racine@intier.com>
                          UNIX Eng.)" cc:
                          <thomas.sjolshag Subject: Re: verify AdvFS domain - fileset is corrupted
                          en@>
                                                                                                                                          
                          05/11/2004 13:02
                          Please respond
                          to "Sjolshagen,
                          Thomas (Tru64
                          UNIX Eng.)"
                                                                                                                                          
                                                                                                                                          

    Try fixfdmn.

    If that doesn't work you can either try "salvage" (to save any
    customizations onto another device) and/or it's - unfortunately - time
    to dig out the backup tapes...

     // Thomas

    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          Bob Harris UBPG
                          <harris@> To: Shaun.Racine@intier.com
                                                   cc:
                          05/11/2004 13:26 Subject: Re: verify AdvFS domain - fileset is corrupted
                                                                                                                                          
                                                                                                                                          

            man fixfdmn

    >
    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          "Smith, Lawrie"
                          <Lawrie.Smith@> To: Shaun.Racine@intier.com
                                                   cc:
                          05/11/2004 13:20 Subject: RE: verify AdvFS domain - fileset is corrupted
                                                                                                                                          
                                                                                                                                          

    Have you tried /sbin/advfs/fixfdmn?

    Kind regards
    Lawrie Smith
    Tru64 UNIX Systems Manager
    Capita Technical Services
    West Malling

    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          "LaVelle, Bryan"
                          <bryan.lavelle@> To: <Shaun.Racine@intier.com>
                                                   cc:
                          05/11/2004 13:34 Subject: RE: verify AdvFS domain - fileset is corrupted
                                                                                                                                          
                                                                                                                                          

    Have you tried fixfdmn? See the man page. There is also a utility called
    salvage available from the HP support center, but it's a last ditch effort
    and you need lots of extra disk space to use it.

    Hope you have taken a 0 level vdump some time recently...

    Regards,

    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          Paul Crittenden
                          <crittend@> To: Shaun.Racine@intier.com
                                                   cc:
                          05/11/2004 14:09 Subject: Re: verify AdvFS domain - fileset is corrupted
                                                                                                                                          
                                                                                                                                          

    YOu could try fixfdmn.

    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          "John L. Lanier"
                          <jllanier@> To: <Shaun.Racine@intier.com>
                                                   cc:
                          05/11/2004 14:44 Subject: RE: verify AdvFS domain - fileset is corrupted
                                                                                                                                          
                                                                                                                                          

    Hello,

    I would recommend "fixfdmn" if you haven't tried this already.

    #/sbin/advfs/fixfdmn root_domain

    If this doesn't work, not much else but to recreate the filesystem.

    Regards,

    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          "Martin Roende
                          Andersen" To: <Shaun.Racine@intier.com>
                          <martin.roende@> cc:
                                                   Subject: SV: verify AdvFS domain - fileset is corrupted
                          07/11/2004 21:22
                                                                                                                                          
                                                                                                                                          

    Other interesting tools are ..

    /sbin/advfs/fixfdmn

    /sbin/advfs/salvage ( last resort , read man page )

    And finally force a read only mount and take a backup if you dont have any,
    check if it is a faulty disk/controller.

    Check up OS patch level as well...

    Regards Martin R. Andersen

    #

    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          "Collins, Robert
                          (HPS Escalation To: <Shaun.Racine@intier.com>
                          Team)" cc:
                          <robert.r.collin Subject: RE: verify AdvFS domain - fileset is corrupted
                          s@>
                                                                                                                                          
                          08/11/2004 21:19
                                                                                                                                          
                                                                                                                                          

    Shaun,
     Try either fixfdmn or if all else failes you can run salvage but
    you will need storage at least as big as the problem domain to do this.
    Both utiles can be found in the /sbin/advfs directory.

    Good Luck,
     Bob

    Bob Collins
    Hewlett Packard
    Global Solutions Engineering
    robert.r.collins@hp.com

    #

    ----- Forwarded by Shaun Racine/MISGroup on 17/11/2004 15:11 -----
                                                                                                                                          
                          Derek Haining
                          <Derek.Haining@> To: "'Shaun.Racine@intier.com'" <Shaun.Racine@intier.com>
                                                   cc:
                          10/11/2004 06:30 Subject: RE: verify AdvFS domain - fileset is corrupted
                                                                                                                                          
                                                                                                                                          

    Shaun,

    I suggest examining the domain using the "fixfdmn" utility. Check the man
    page for the exact syntax.

    My personal suggestion is to run fixfdmn with the "-n" flag at first.
    (This
    should be the "no write" option.) Then check the log file that fixfdmn
    wrote
    to see what kinds of errors it reported finding. If I recall correctly,
    fixfdmn
    should tell you if it thinks it cannot fix a certain problem.

    Differences between fixfdmn and verify are:

    verify does a "mount" of the domain/fileset in order to check it. Thus if
    there
    is corruption that causes the kernel a problem, you cannot fix it with
    verify.

    fixfdmn does not do an AdvFS mount of the domain. Instead it opens each of
    the
    volumes of the domain as block devices. It does all of its checks in user
    mode,
    so it cannot crash the system. In addition, it is able to fix some errors
    that
    the kernel cannot handle.

    Good luck!

     -Derek Haining


  • Next message: Bugs: "SUMMARY: pg_nreaders going negative"