Re: BACKUP locks up a directory?

AEF wrote:
Dave Froble wrote:
AEF wrote:
Richard B. Gilbert wrote:
AEF wrote:
I was doing an image backup of an 18-GB disk. During this backup one of
my market servers was running the end-of-day job. At the end of the job
it copied certain files created the same day to another VAX system. A
total of 24 files are normally copied. These range in size from a few
blocks to a couple thousand. Anyway, the first 15 files were copied
okay but the next 15 failed with the following error (actual sample
Uh, that should have read "...the next 9 failed..."

from the log file):

%COPY-E-OPENOUT, error opening NODE16"FT
password"::FT_STORAGE_DISK:[FT.ARC.EQUITIES]FTOATTRN060929.SEQ;0 as
output
-RMS-E-FLK, file currently locked by another user
%COPY-W-NOTCOPIED, _NODE2$DKA400:[FT.DAT]FTOATTRN060929.SEQ;1 not
copied

I assume that the "other user" was the backup job.

The DCL command was

$ COPY/LOG FTDAT:FT*060929*.*;0/EXCLUDE=(FTGVT*.DAT,FTSYS*.DAT)
NODE16"FT XXXXXXXX"::FT_STORAGE_DISK:[FT.ARC.EQUITIES]*.*;0

So I got 15 success messages, followed by 9 error messages like the one
above, followed by the summary message.

I aborted the backup job at this point and re-ran the EOD job. It ran
fine and copied all 24 files successfully. I looked at the tape and
sure enough, the backup job had just started copying files from the
[FT.ARC.EQUITIES] directory.

Some more possibly relevant information: NODE2 is running VMS v6.1.
NODE16 is running VMS v6.2. The directory [FT.ARC.EQUITIES] contains
about 24000 files. The EQUITIES.DIR file is 2751 blocks in size.

NODE2 is a MicroVAX 3100 Model 95. NODE16 is a MicroVAX 3100 Model 80.

Is this normal BACKUP behavior? The backup takes so long that it is
difficult to avoid the EOD job. I've done this before several times and
never had this problem. I assume I am a victim of bad timing. But is
this normal, that a "hot backup" can lock up directories and/or files?
Is there a better way to do this?

Thanks.

AEF

With a directory 2751 blocks in size, it's no wonder things are a little
slow! Later versions of VMS are not quite so sensitive but at VMS V6.2
any file added to or deleted from an oversize directory could create an
incredible amount of overhead. Directory entries are stored in
alphanumeric order. When an entry is inserted in or deleted from a
directory, blocks get shuffled up or down to make room or reclaim space.
It's no big deal if you keep your directories under 100 blocks.

If you really need to have all 24,000 files on line, I'd suggest
spreading them over a LOT more directories. Maybe a directory for 2006,
one for 2005, 2004, . . . or a directory for each month or whatever.
Thanks for your quick response.

Performance is not really an issue.
You indicate that the BACKUP performance is an issue. Your statement is
wrong.

Well, it is and it isn't. I have A LOT OF TROUBLE with these blasted
TLZ tape drives. (I'm the one who wrote "I Lost It with the Tape
Drive".) Currently, I'm happy if I just end up with tapes that I can
read back later. Sure, I'd like it to go faster, but that's the least
of my problems. In fact, I switched from DDS-2 tapes to DDS-1 tapes on
the hope that it would it would greatly increase the likelihood of
making tapes that are readable when I need them later. Until this
episode, it seemed to be working, which is ANOTHER problem I'm having.
The stupid drive is having trouble verifying the tape it just made!
Arrrggh. The problem here is that the backup somehow screwed up the
file-copy operation requiring me to rerun the EOD job (which because of
other screwy stuff forces me to manually run and adjust the report job,
but never mind that now) and I'd just like to understand better what's
going on.

4 MM DAT was a crap shoot since the day it first came out. That's just the way things are. There have been many posts in the past, with the usual recommendation to pick up a DLT drive. DLT4 and subsequent models store significant data, and suffer many more passes than 4 MM DAT. I understand new tapes for the older drives can be an issue.

The files normally copy over rather
quickly even though the directory is so huge. I suspect that BACKUP
started reading the .DIR file in the middle of the copy operation.
Could it be that the copying instigated a .DIR-file expansion and the
shuffling of the directory data screwed up the copy? Or does it have
something to do with BACKUP? (If I needed to delete selected files,
that would certainly go rather slowly!!!)

Actually, I suppose the backup would run faster if I combined each
day's files into BACKUP save sets. I actually do that before I archive
them to CD's, but I prefer to have the individual files available
online, at least for stuff this year (I need the current year's files
in order to run occasional usage reports).

Re spreading files over more directories -- I guess that would greatly
reduce the odds of this problem happenning again. But it would be more
work than it's worth.
No, it's not. If you have any method of identifying files by time, then
the work is rather trivial. Retrival may also require some work, but

Retrival? What is that?

defining a logical with a list of values handles that. A directory that
large is obscene even on VMS V7.2 and later. On V6.2 it's very much
worse. You've been given excellent advice and you've decided to ignore
it. Why did you ask?

Still pissed off at me, huh, aren't you Dave? Anyway, I didn't ignore
the answer, I just commented on it.

?????????

When was I ever upset with you. I know I'm getting old, crotchety, and senile, but the only person I remember that caused frothing at the mouth was JF, and now I don't let him bother me.

Large directories on V6.2 are a greater problem than just what you see on the surface. It's not just deletions that cause problems, though those are rather visable.

Anyway**2, it's not worth the trouble because I've been doing this for
the last 5 years or so and have only once had this problem. It is far
easier to rerun and end-of-day once every 5 years than to rewrite the
EOD code to deal with a new-fangled directory structure and QA it and
get management approval and install it and debug it. I'd also have to
rewrite some other code that moves each day's files into a single save
set for transference to a CD-R. I'd also have to rewrite my
usage-report command procedures. OK, that wouldn't be too hard given
logical names.

Yes, the directories are rather large. But it's not really a problem
because I am the only one who ever uses these files. If I had to delete
them, there are several tricks available to speed it up. WIldcard
lookups are a bit slow, but I can do other things while they run. Why
do I even have these file online? I was told that they should be made
available just in case someone needed them for auditing purposes or
checking something or whatever. Guess how many times that's happened?
ZERO! And that covers at least 5 years. I'm also supposed to archive
them all to CD's, but that's a real pain, and I've read that CD-R's may
not last more than a few years. Taping is far easier except that the
TLZ tape drives make write-probably/read-maybe tapes!

Your comments may be applicalbe in other situations, but I really
didn't want to go into so much deatil.

Why did I ask? I didn't ask the question whose answer you speak about.
I asked if BACKUP locking up a directory is normal behavior or if
something else is happening. 7

I'd be real surprised if BACKUP locked a directory, unless you used the /RECORD switch (I think that's the one that sets the last backup date). It's mainly a read only operation. However, if it was reading, and another process wanted exclusive access, that might be an issue. Don't know enough about the details to be more specific.

Not picking on anyone, just emphasizing that very large directory files are a serious problem.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@xxxxxxxxxxxxx
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
.

Relevant Pages

  • Re: Offsite backup options
    ... I like Exabyte VXA Drives for this. ... are sized and priced for what you need, and if you outgrow the tapes you can ... Any backup to a NAS, storage server, home server, or whatever seems like ... Can only do critical data ...
    (microsoft.public.windows.server.sbs)
  • Re: Anyone using Iomega REV for SBS Backup?
    ... are there any issues/gotchas with using SBS Backup for restore ... We use Rev drives on lots of our SBS installations after experiencing lots ... install any of the Iomega drivers and just use Filestreamer RM. ... not so costly for DAT but Travan tapes are very expensive. ...
    (microsoft.public.windows.server.sbs)
  • Re: Need backup hardware suggestions
    ... > I am exploring various solutions for reliable backup for my exclusively ... > Tapes or external USB harddisks? ... I happen to prefer Exabyte VXA-2 tape drives, ... Since my initial installation of the VXA-1 drive, ...
    (comp.os.linux.misc)
  • Re: Need backup hardware suggestions
    ... > I am exploring various solutions for reliable backup for my exclusively ... > Tapes or external USB harddisks? ... I happen to prefer Exabyte VXA-2 tape drives, ... Since my initial installation of the VXA-1 drive, ...
    (comp.os.linux.hardware)
  • Re: BACKUP locks up a directory?
    ... You indicate that the BACKUP performance is an issue. ... I'm happy if I just end up with tapes that I can ... understand new tapes for the older drives can be an issue. ... Still pissed off at me, huh, aren't you Dave? ...
    (comp.os.vms)