Re: How many files can you have in a VMS directory without performance problems? performance problems? performance problems?

norm.raphael_at_metso.com
Date: 08/17/05 

Date: Wed, 17 Aug 2005 09:52:51 -0400

Jeff Cameron <roktsci@comcast.net> wrote on 08/17/2005 09:10:04 AM:

> On 8/17/05 3:46 AM, in article
> 1124275575.684487.287580@z14g2000cwz.googlegroups.com, "RAQ"
> <_raq_@yahoo.com> wrote:
>
> > Hi
> >
> > Everytime I ask HP about
> > "How many files can you have in a VMS directory without performance
> > problems?"
> >
> > they send me an article that is about 10-15 years old for VMS 5.2 or
> > even 1.0, that describes that more than 300 files in a catalog is
> > bad...
> >
> > I have a feeling that <25.000 files works ok (no major complains), but
> > when we reach 100.000 files it gets bad - and with 500.000 it gets
> > really nasty :)
> >
> >
> > Does anyone know the real numbers? Or how I can calculate them?
> > Its for VMS 7.x + EVA SAN, and is mainly for file creation.
> >
> >
> >
> > Thanks
> >
> There are basically three things that can cause directory size
performance
> issues.
> 1. SYSGTEN parameter ACP_DIRCACHE.
> 2. The speed of your disk.
> 3. The speed of your processor.
> The reason is that VMS keeps the directory entries sorted, and quite
often
> the adding and deleting of entries can cause the entire .DIR file to be
> flushed from cache and completely re-written.
>
> As mentioned by Chris Blackburn this is particularly noticeable when
> deleting files. When you delete files from a directory, it is done in
> alphabetical order, or from the top of the list. Depending on how much of
> the .DIR file is cached (ACP_DIRCACHE) when the cache needs to be
flushed,
> this causes the entire .DIR file to be rewritten to disk. Since entries
were
> removed from the top, and the directory entries are all shifted up and
each
> block of the directory file has now changed because the entries were
shifted
> up.
>
> This issue can be shown by my command procedure REV_DEL.COM
> (http://support.mti.com/VMS2005cd/supportfiles/rev_del.txt)

Isn't there a penalty paid for activating the DELETE.EXE image for every
file,
instead of handing it a list, or a wildcarded list?

(Oh and you spelled deleting as deleteing in one of your comment lines :-|
.
> which deletes files in the reverse order, that is from the bottom of the
dir
> file. That has the effect of not having to rewrite the entire .DIR file
with
> each cache flush, because the only portion of the DIR file that has
changed
> is the end. So if you are deleting files (even a portion of the files)
from
> the bottom or reverse order will take much less time because the upper
> blocks will not be changing with each cache flush.
>
> If you have a large .DIR file and you delete files in the normal fashion
> with the /LOG qualifier, you will see a hand-full of files being deleted,
> then a pause, then another hand-full, and another pause, over and over
again
> with each pause being when the .DIR file is being flushed back to disk
> completely re-written. As the .DIR file gets smaller and smaller the
flush
> operation becomes shorter and shorter. If you are using MONITOR/FILE you
can
> see your directory cache hit rate toggle back and forth from 100% to low
> percentages (again dependent on ACP_DIRCACHE settings). However deleting
in
> the reverse order, you will not see these pauses, because the .DIR file
is
> only being truncated rather than being rewritten.
>
> SO back to your question, what is the number? It depends mainly on what
kind
> of actions you are doing and if the changes are additions or deletions
and
> where they are being inserted/removed from the .DIR file. If the changes
are
> made near the end of the .DIR file then the consequences are low. If the
> changes are at the top, then when a cache flush occurs more blocks must
be
> rewritten to disk. Given this information you can see that increasing any
of
> the the three items listed here (in this order of most effective first)
can
> help you forgo the cost of directory flushing.
>
> 1. ACP_DIRCACHE size.
> 2. Disk speed.
> 3. Processor speed.
>
> Jeff Cameron
>

Relevant Pages

  • Re: How many files can you have in a VMS directory without performance problems?
    ... The reason is that VMS keeps the directory entries sorted, ... the adding and deleting of entries can cause the entire .DIR file to be ... which deletes files in the reverse order, that is from the bottom of the dir ... blocks will not be changing with each cache flush. ...
    (comp.os.vms)
  • Re: NIC deleting in the Registry
    ... Registry etc. ... The deleting of it from the Device Manager is leading to an Error something ... are these keys somehow locked and should these deletions maybe be made ... unnecessary/unused entries related to these, ...
    (microsoft.public.windowsxp.help_and_support)
  • Re: Cant send smtp email
    ... If Kath's suggestion doesn't work and you have ZoneAlarm, try deleting all ... entries in ZA's Programs list (r-click the tray icon). ...
    (microsoft.public.windowsxp.general)
  • Re: Forcing a selection
    ... acompany names and contains the Town the company is in. ... Users have the ability to delete entries that aren't theirs by selecting ... How can I force the user to select both columns when deleting. ...
    (microsoft.public.excel.programming)
  • Re: How to Clear Old Items From the Run Command
    ... A link provided by Ramesh suggests deleting the Content.IE5 folder, ... When I enter a "w" in the Run space, this list of six entries drops down. ...
    (microsoft.public.windowsxp.general)