Re: Slow Filesystem I/O

From: Bill Todd (billtodd_at_metrocast.net)
Date: 04/24/05 

Date: Sun, 24 Apr 2005 15:33:08 -0400

Alan Greig wrote:
> Bill Todd wrote:
>
>
>>Any operating system at all serious about its future would have fixed
>
>
>>such default behavior a decade or more ago: looking like an utter
>
> slug
>
>>when compared with the default behavior on Windows or Unix,
>
> especially
>
>>while providing no stronger guarantees that the data has actually
>
> made
>
>>it onto the disk as compensation, is absurd.
>
>
> Recall Bill that VMS did fix this problem a decade ago with
> Spiralog/TNFS/Files-64.

And then fairly quickly rescinded support for that fix. That's simply
more evidence that about a decade ago serious support for VMS's future
started grinding to a halt, though inertia allowed considerable
additional significant development to come to fruition before that
process completed with the diversion of most remaining effort into the
Itanic port.

  Sure it had problems but rather than fix or
> re-implement the programmers were sacked during downsizing and the
> product dropped. With that file system in place a simple write of an
> 80MB file could indeed complete almost instantly. I ran a full-feed
> news-server (ANU News) with Spiralog field-test for some time and met
> some of the developers.

My (admittedly vague) recollection is that (for obvious reasons)
Spiralog did not change *default* file system behavior in any
application-visible way, but rather made a considerably richer set of
options available (some of which may, of course, have been options to
change the default behavior manually). So I don't understand how
Spiralog itself could have affected RMS's default performance nearly
that much (e.g., when RMS specified that its internal buffer should be
written to disk, it had the right to assume that it indeed had been
written when the QIO completed, so Spiralog would have had to respect
that right by performing a synchronous disk write: yes, it could have
combined that with other data needing to make it to disk, but the bottom
line would remain that it would take at last one disk revolution to
destage each 8 KB RMS buffer, which is nowhere nearly equivalent to
making the disk's entire sequential bandwidth available to that single
application).

Now, it certainly would have been possible to change *RMS* (probably
only in relatively minor ways) such that it could tell QIO "Hey, just
take this small buffer off my hands so I can reuse it, and if I ever
really care when it actually gets to disk (e.g., the application
requests a FLUSH or the file is closed) I'll give you a nudge". If such
changes were implemented, then indeed it would have solved much of even
the default-behavior performance problem.

- bill

Relevant Pages

  • Re: .cmd file and Chkdsk
    ... Your computer will work perfectly with or without unused security descriptors. ... "Bill" wrote in message news:%23EMxFaHFEHA.1560@TK2MSFTNGP12.phx.gbl... ... > errors in the Volume Bitmap" "Windows found problems with the file system" ... >> A disk check has been scheduled. ...
    (microsoft.public.windowsxp.general)
  • Re: [PATCH] jbd: dont abort if flushing file data failed
    ... buffer has an error. ... data to the disk, should we not mark the file system readonly by aborting ... If we make the file system read-only on a file data write error, ... So some people need this patch. ...
    (Linux-Kernel)
  • disktab for VPC Disk
    ... I'm trying to optimize the disk access of NEXTSTEP on Virtual PC 5.0.4. ... Maximum size of NeXT file system is 2GB. ... programs in the initial 8 sectors of the device. ... The number of cylinders per cylinder group in a file system. ...
    (microsoft.public.mac.virtualpc)
  • [PATCH 3/3] Add ext3 data=guarded mode
    ... buffer onto a list of things that must be written before a commit. ... a workqueue where the real work of updating the on disk i_size is done. ... When we start tracking guarded buffers on a given inode, ... and it also takes a reference on the buffer head. ...
    (Linux-Kernel)
  • Re: [PATCH RFC] ext3 data=guarded v6
    ... Fixup locking while deleting an orphan entry. ... Fixup O_DIRCECT disk i_size updates ... buffer onto a list of things that must be written before a commit. ... When we start tracking guarded buffers on a given inode, ...
    (Linux-Kernel)