Efficiency Question: Large Arrays vs. Indexed Files on Alphas

From: Cheryl Hoefelmeyer (hoefelmeyer_at_hotmail.com)
Date: 11/20/03 

Date: 20 Nov 2003 14:38:14 -0800

We have a GS80 and an ES40, not clustered, each running OpenVMS 7.3. I
am writing a program in Basic that will run on each box, and I have a
question regarding which of the following would be most efficient, in
general.

The program will operate on three very large files, and at one point I
can either
(1) decide to read some fields of some records into an array and
search for information sequentially through the array, or
(2) read the information into another file and access it each time via
single read.
The number of elements or records in this circumstance is not expected
to exceed 10,000.

Elsewhere in the program, I can either
(1) maintain records that will eventually be written to an output file
in an array because they may need to be updated some number of times
before a final record is written, or
(2) write the first record to the output file and just update each
time it is necessary.
The first option would call for reading sequentially through a very
large array to find the proper record to update each time a new record
is added. The second calls for 1-3 file operations per each record
added. The number of records maintained here is on the order of
1,000,000.

So, for each of these, which is the best option in general? Years ago,
I/O on DEC boxes was such a bottleneck that the array option would
generally be the faster route, but I’m no longer sure that is
such a big issue now.

Loading