Re: Performance problem (I/O)

From: Bob Harris (harris_at_zk3.dec.com)
Date: 03/21/04

  • Next message: Nikola Milutinovic: "Re: TRU64 C++" 
    Date: Sun, 21 Mar 2004 04:48:56 GMT

    In article <c3ikhh$j9g$1@jeeves.eng.abbnm.com>,
     peter@abbnm.com (Peter da Silva) wrote:

    > In article <harris-214E79.19555619032004@cacnews.cac.cpqcorp.net>,
    > Bob Harris <harris@zk3.dec.com> wrote:
    > > In article <c3g297$19vl$1@jeeves.eng.abbnm.com>,
    > > peter@abbnm.com (Peter da Silva) wrote:
    > > > for the case of small files (<8K)
    > > > the write should be completed and the close performed before anything hits
    > > > the disk... I wouldn't expect actual disk writes for the 8k blocks.
    >
    > > If UFS serves your needs for small files, then by all means use UFS.
    > > Both file systems are fully supported.
    >
    > Um, yeh, can you say whether AdvFS actually does these redundant writes
    > or not, though?

    I'm not sure I understand the question. I think, based on what you have
    extracted above, you are wondering if AdvFS defers the 8K allocation
    until after the frag has been created, and just what I/O really occurs.
    Is this correct?

    Anyway, AdvFS will allocate the files last 8K page before it knows if
    the file is going to be closed and the last 8K turned into a frag. The
    allocation will involve modifications of the SBM (Storage Bit Map), the
    insertion of the storage extent information in the file's Mcell extent
    map (allocation might have been part of a larger allocation for the
    file) and a transaction will be written to the log. The SBM and Mcell
    updates will be done as lazy writes, so that multiple updates to the
    same pages may occur. The log transaction will get written to disk,
    again in a bit of a lazy mode, but not nearly as lazy as the SBM or
    Mcell writes.

    The data for the last page may or may not get written to disk. Depends
    on how long between the last write and the closing of the file. My
    guess is that creating a small file is done, create, write, close in
    short order, so I would guess that the data in the last 8K allocation of
    the file is still sitting in the cache.

    The _EXACT_ order that some of the following happens may be wrong, but
    what happens is essentially correct.

    When the close happens and the decision is made to frag the last 8K of
    the file, a transaction is started. A frag of the correct size is
    allocated from the frag file (ls -l /mount_point/.tags/1). If the frag
    file does not have a free frag of the correct size, then more storage is
    allocated to the frag file (SBM updates, frag file's Mcell updated,
    information added transaction). This involves updating free pointers in
    the frag file and adding information to the transaction. The last 8K of
    the file is removed from the file's Mcell, and the pointer to the
    allocated frag is stored in the file's Mcell (all of this gets more
    information added to the transaction). The data from the last 8K is
    copied to the allocated frag. Because the frag file is considered
    metadata, all of the data is also added to the transaction. The 8K is
    freed by clearing its bit in the SBM (and more information is added to
    the transacation). The file's Mcell, the SBM page, the frag in the frag
    file will all eventually get flushed to disk in a lazy fashion and if
    there are other modifications to the affected pages before that happens
    this will be saved write I/Os. The transaction will be flushed to the
    log, it may not happen right away, and some additional transactions may
    go along with it, but it will go in short order.

    So not every action results in its own I/O, however, there are factors
    that can detract from the idea of maximum caching and lazy updates of
    the metadata. In the case of a well aged file system where files have
    been created, created, deleted, etc..., the available free space may not
    be contigous. The free Mcells for new files may not be next to each
    other. The free frags for fragging files may not be next to each other.
    So the ability to have multiple updates to the same page of metadata
    storage for multiple file creates may not occur. So while the I/O to
    flush a modified metadata page may not happen right away, it may not get
    any company during the flush either. So it is still going to be real
    I/O that will affect I/O bandwidth.

    Possible real I/Os. log transaction flush (best bet for always being
    able to enjoy company from other operations). Flush the SBM (better
    than average that multiple allocations will happen on the same pages
    before a flush, and at least the allocation of the last 8K and the
    freeing of it may occur before the flush happens). The flush of the
    Mcell (on a well aged file system, this is more than likely to not have
    company when it is flushed). The flush of the frag (on a well aged file
    system, this is unlikely to have company when it is flushed).

    And I didn't discuss reads to bring metadata into memory. On a well
    aged file system the avalable Mcells and frags in the frag file may not
    already be in memory, so reads may occur to get them.

    So on a well aged file system, say we average 1 seek/read and 1
    seek/write for metadata, and 1 seek/write because the file data has to
    be written to disk eventually. So frag'ing the file on close might
    involve 3 I/Os (the data portion had to occur eventually). And as I've
    detailed above, it could be worse if there is very little sharing of the
    I/O by multiple operations. And this does not include the operations
    need to create the file in the first place and allocation the initial
    storage.

    But because of the dynamic nature of the system, and not knowing how
    much sharing can occur with the same I/O, I can not tell you exactly how
    much I/O will be needed to frag a file.

                                            Bob Harris

  • Next message: Nikola Milutinovic: "Re: TRU64 C++"

    Relevant Pages

    • Re: sequential disk read speed
      ... closer to the theoretical optimum for I/O. ... of allocation. ... I/O for the transaction logs. ... I agree that if the disk subsystem caches a track then a smaller unit ...
      (comp.databases.theory)
    • Re: Performance problem (I/O)
      ... This could be due to the way the last 8K allocation ... For space efficency, files under 150K tend ... > in space can be obtained by turning the last 8K into a frag, ... I wouldn't expect actual disk writes for the 8k blocks. ...
      (comp.unix.tru64)
    • Quantcast