Re: Performance problem (I/O)

From: Bob Harris (harris_at_zk3.dec.com)
Date: 03/19/04

  • Next message: SANFACE Software: "Shareware: txt2pdf 7.2" 
    Date: Fri, 19 Mar 2004 02:26:47 GMT

    In article <ff81d3b0.0403180401.73357d13@posting.google.com>,
     paltskallen@hotmail.com (Fredrik) wrote:

    > Thank you for the information, all the seeks must take a long time.
    > But I created a new domain with exactly the same configuration och
    > copied all the files to that domain (with tar) to run some more
    > performancetests, but tar'ing a directory on the old domain takes
    > about 10 minutes but on the new it only takes ~1 minute!!!!!

    When you restored the file system using tar, you placed all the files in
    the exact order that you would then back them up. You positioned the
    metadata for each file next to the file that would preceed it. As a
    result you maximized your backup performance.

    But it is unlikely that you will actually create 8 million files in the
    exact same order that they will be backed up. Generally, files get
    created and deleted in different directories in a random order. This
    means reusing metadata from a from some other deleted file. It means
    reusing storage for the files from a deleted file.

                                            Bob Harris

    > defragment on the old domain says:
    > defragment -nv store_domain
    > defragment: Gathering data for domain 'store_domain'
    > Current domain data:
    > Extents: 114860
    > Files w/extents: 90821
    > Avg exts per file w/exts: 1.26
    > Aggregate I/O perf: 93%
    > Free space fragments: 44246
    > <100K <1M <10M >10M
    > Free space: 5% 66% 22% 7%
    > Fragments: 12065 29918 2245 18
    >
    > This doesnt look all to bad or??
    > Can any explain the huge difference in performance??
    >
    > Bob Harris <harris@zk3.dec.com> wrote in message
    > news:<harris-50A08F.20371715032004@cacnews.cac.cpqcorp.net>...
    > > In article <ff81d3b0.0403150527.2b6004aa@posting.google.com>,
    > > paltskallen@hotmail.com (Fredrik) wrote:
    > >
    > > > Hello!
    > > >
    > > > We have a advfs filesystem with ~ 8 million small files and it is
    > > > still growing.
    > > > This all is in a SAN, HSG80 with striped disks.
    > > >
    > > > The weekly networker fullbackup takes about 20 hours and it seems that
    > > > it is the os that is waiting for io.
    > > > It is also lsm mirrored between two different SAN's
    > > > The tar command takes about the same time so it seems that it not a
    > > > networker problem.
    > > >
    > > > output from collect:
    > > > # DISK Statistics
    > > > #DSK NAME B/T/L R/S RKB/S W/S WKB/S AVS AVW ACTQ WTQ
    > > > %BSY
    > > > 0 dsk19 - 136 1091 176 1467 5.27 30.76 1.65 3.87
    > > > 76.69
    > > > 1 dsk7 - 136 1097 174 1457 4.94 25.20 1.54 3.14
    > > > 79.09
    > > >
    > > > Anybody has any tips, clues how to speed it up or what the problem is?
    > >
    > > That seems about right. 8 million files means most likely a minimum of
    > > about 8 million seeks, with each seek averaging about 8 milli-seconds.
    > >
    > > Actually, you are possibily getting faster than 8 milli-seconds per
    > > seek, but figure that there is more than 8 million seeks. There are
    > > seeks to lookup the filename (most likely a lot of this is cached), seek
    > > to get the metadata for the file being opened (but in some cases the
    > > metadata page from a previous file open is cached). Seek from the
    > > metadata to the user data when you read the small file. If the small
    > > file was fragmented, it might be in more than one 8K page, or 8K page
    > > and a less than 8K frag in the frag file (might get some caching if a
    > > frag shared the same page and was still in the cache). Seek for each
    > > non-contiguous extent. And because you opened the file, the file
    > > modification time was changed, so seek back to the metadata and update
    > > the time stamp. And because there is a metadata modification, the
    > > metadata change is transaction logged, so there will be a seek to the
    > > log (this is a lazy update, so multiple updates might be consolidated
    > > into a single write to disk).
    > >
    > > If you don't care about access times, you could use the noatimes mount
    > > option.
    > >
    > > Place the log file on its own volume (switchlog). Create a volume a bit
    > > larger than the log file, and then addvol and switchlog.
    > >
    > > If your files are typically greater than 8K, but less than 100K to 150K,
    > > and are not typically an even multiple of 8K, then if you are not
    > > concerned about the extra disk space, you could disable the frag file so
    > > that the minimum file size if 8K and grows in mulitples of 8K (man
    > > chfsets on more recent versions of AdvFS, otherwise it is a dbx patch
    > > the kernel method; but it only affects new files or files that are
    > > extended; existing files remain frag'ed). NOTE: This can increase the
    > > amount of storage a lot. For example if you have mostly 1K files you
    > > would increase your storage needs 7 fold by disabling frags. But if
    > > your files are more like 100K, then maybe only 10% more space.
    > >
    > > I'm sure someone else might have ideas.
    > >
    > > Bob Harris

  • Next message: SANFACE Software: "Shareware: txt2pdf 7.2"
    • Quantcast