Re: Slow Filesystem I/O

From: Dan Foster (usenet_at_evilphb.org)
Date: 04/24/05

  • Next message: Patrick MOREAU, CENA Athis, Tel: 01.69.57.68.40: "Re: Slow Filesystem I/O" 
    Date: Sun, 24 Apr 2005 16:59:49 +0000 (UTC)

    In article <1114356068.260432.193010@o13g2000cwo.googlegroups.com>, Alan Greig <greigaln@netscape.net> wrote:
    >
    > Recall Bill that VMS did fix this problem a decade ago with
    > Spiralog/TNFS/Files-64. Sure it had problems but rather than fix or
    > re-implement the programmers were sacked during downsizing and the
    > product dropped. With that file system in place a simple write of an
    > 80MB file could indeed complete almost instantly. I ran a full-feed
    > news-server (ANU News) with Spiralog field-test for some time and met
    > some of the developers. The project was based at DEC Livingston
    > (Edinburgh) and I still have the t-shirts.

    That's a shame (Spiralog). Wonder why then-DEC dropped it if it was
    pretty good? There's some rumours of boundary conditions presenting
    significant engineering work making it unpalatable to develop further?

    Note to self: a quick primer on filesystem writes -- three major policies:

            a) Write-through caching -- app writes to fs; fs doesn't return
               until all data blocks has been successfully written to disk.

               Safest but quite slow for obvious reasons. Default on OpenVMS.

            b) Write-back caching -- app writes to fs; fs returns as soon as
               data is in cache, even if cache has not yet flushed to disk.

               Very unsafe -- very good performance but if interrupted (e.g.
               power failure) at wrong time, filesystem data corruption.

            c) Write-behind caching -- app writes to fs; fs reorders data
               as necessary in order to guarantee that data will never be
               corrupted even if interrupted mid-write, and also retains
               performance properties of write-back caching.

               Provides performance of write-back with safety of write-through.

    The performance you mentioned sounded like it used write-back or some
    variant. So being curious, I checked a paper on Spiralog from:

    http://research.compaq.com/wrl/DECarchives/DTJ/DTJM04/DTJM04P8.PS

    ...written by some of the key engineers on the project.

    I didn't have a good PostScript viewer so I ran it through ps2pdf on a
    non-OpenVMS machine with excellent results and then viewed the resulting
    PDF file.

    Apparently, Spiralog utilized write-behind caching which gained the
    benefits of write-back performance and had the safety of write-through.

    I liked the engineering considerations put into it such as knobs to set
    policy on a per-file basis as well as at other layers (e.g. RMS on
    OpenVMS).

    Did any remnants of the work make it into any component of subsequent
    OpenVMS releases?

    -Dan

  • Next message: Patrick MOREAU, CENA Athis, Tel: 01.69.57.68.40: "Re: Slow Filesystem I/O"