Re: Fwd: Abyssmal dump cache efficiency

On 2007-Feb-20 02:47:00 -0500, Zaphod Beeblebrox <zbeeble@xxxxxxxxx> wrote:
On 2/17/07, Peter Jeremy <peterjeremy@xxxxxxxxxxxxxxxx> wrote:
I've tried modelling a unified cache along the NetBSD line and there
appears to be a massive improvement in cache performance. It's unclear
how much of an improvement this will give in overall performance but
not physically reading data from disk must be faster than reading it.

This squares perfectly with my recent observation that while runing some
combination of "dump | restore" that the dump disks incur 2 to 3 times more
I/O (reading) than the restore disks. Now... for "performance" I was using
the cache function --- maybe the cache is actually a detriment.

The limited testing I've done suggests that 32MB cache gives you a
10-20% improvement in dump speed. This would heavily dependent on
the disk I/O performance - a slow CPU running PIO might be better
off without caching.

I've found that you do get a worthwhile improvement in dump|restore
performance by introducing a large (10's of MB) fifo between them.
This helps reduce synchronisation between dump and restore (so that
dump can continue to read whilst restore is busy writing a batch of
small files and vice versa). There's a suitable port but I can't
recall the name because I wrote my own.

I believe it would be worthwhile creating a todo item to investigate
this more thoroughly.

Note that I think that fixing this is a weekend job, rather than a SoC
project.

--
Peter Jeremy

Attachment: pgpCJPvBa4GFA.pgp
Description: PGP signature

Relevant Pages

  • Re: USB2.0 throughput
    ... Both my setups are reading a file from the USB HDD device. ... reading speed cannot be speed-up by a cache. ... It seems that the wince system is not fully loading the USB ...
    (microsoft.public.windowsce.embedded)
  • Re: Request: I/O request recording
    ... > file, ready for optimized reading. ... Linux caches disk data on a per-file basis. ... we don't go looking for the same disk blocks in the cache of /dev/hda1. ... send the line "unsubscribe linux-kernel" in ...
    (Linux-Kernel)
  • Re: Easy - if you know how.
    ... actually writing to disks. ... No read cache, slow write. ... Could I make my system faster by updating all files to 1 second ... This means write access to the hard drive when reading files. ...
    (alt.os.linux.suse)
  • Re: clearing filesystem cache for I/O benchmarks
    ... I'm reading via gmane and my newsreader doesn't make it ... clears kernel page cache and not controller/disk cache. ... based on your advice, I'll instead just create a 2x "dummy" file, ... send the line "unsubscribe linux-kernel" in ...
    (Linux-Kernel)
  • Re: sparse file performance (was Re: Is there a "make hole" (truncate in middle) syscall?)
    ... With a 10MB window, I get 331 MB/s ... reading out of buffer cache and 185 MB/s reading a hole. ... send the line "unsubscribe linux-kernel" in ...
    (Linux-Kernel)