Re: How does disk caching work?

From: Uwe Doering (gemini_at_geminix.org)
Date: 04/17/04

  • Next message: Jim C. Nasby: "Re: How does disk caching work?" 
    Date: Sat, 17 Apr 2004 09:41:19 +0200
To: freebsd-performance@freebsd.org

    Jim C. Nasby wrote:
    > On Sat, Apr 17, 2004 at 01:56:55AM +0400, "Igor Shmukler" wrote:
    >
    >>>Is there a document anywhere that describes in detail how FreeBSD
    >>>handles disk caching? I've read Matt Dillon's description of the VM
    >>>system, but it deals mostly with programs, other than vague statements
    >>>such as 'FreeBSD uses all available memory for disk caching'.
    >>
    >>Well, the statement is not vague. FreeBSD has a unified buffer cache. This means that ALL AVAILABLE
    >>MEMORY IS A BUFFER CACHE for all device IO.
    >>
    >>>I think I know how caching memory mapped IO works for the most part,
    >>>since it should be treated just like program data, but what about files
    >>>that aren't memory mapped? What impact is there as pages move from
    >>>active to inactive to cache to free? What role do wired and buffer pages
    >>>play?
    >>
    >>If file is not memory mapped it is not in memory, is it? Where do you cache it? Maybe I am missing
    >>somewhing? Do you maybe want to know about node caching?
    >
    > What if the file isn't memory mapped? You can access a file without
    > mapping it into memory, right?

    In FreeBSD, file and directory data always exists as VM objects, that
    is, a collection of virtual memory pages. Those that have been accessed
    exist in physical memory (if not recycled due to inactivity), the rest
    is just reservations. That's why it is called "virtual memory".
    Whether these objects get accessed by read()/write() or mmap() depends
    on your application. These system calls are just different userland
    interfaces to the same kernel resource.

    >>When pages are rotated from active to inactive and then to cache buckets they is still retains vnode
    >>references. Once it is in free queue, there is no way to put it back to cache. Association is lost.
    >>
    >>Wired pages are to pin memory. So that we do not get situation when fault handling code is paged out.
    >>
    >>I am not FreeBSD guru so I never heard of BUFFER pages. Is there such a concept?
    >
    > I'm reffering to the 'Buf' column at the top of top. I remember reading
    > something about that being used to cache file descriptors before the
    > files are mapped into memory, but I'm not very clear on what is actually
    > happening.

    The disk i/o buffers you refer to (the 'Buf' column in 'top') are the
    actual interface between the VM system and the disk device drivers. For
    file and directory data, sets of VM pages get referred by and assigned
    to disk i/o buffers. There they are dealt with by a kernel daemon
    process that does the actual synchronization between VM and disks.
    That's where the soft updates algorithm is implemented, for instance.

    In case of file and directory data, once the data has been written out
    to disk (if the memory pages were "dirty") the respective disk i/o
    buffer gets released immediately and can be recycled for other purposes,
    since it just referred to memory pages that continue to exist within the
    VM system.

    Meta data (inodes etc.) is a different matter, though. There is no VM
    representation for this, so for disk i/o they have to be cached in extra
    memory allocated for this purpose. A disk i/o buffer then refers to
    this memory range and tries to keep it around for as long as possible.
    A classical cache algorithm like LRU recycles these buffers and memory
    allocations eventually.

    As usual, the actual implementation is even more complex, but I think
    you got a picture of how it works.

        Uwe 

    -- 
Uwe Doering         |  EscapeBox - Managed On-Demand UNIX Servers
gemini@geminix.org  |  http://www.escapebox.net
_______________________________________________
freebsd-performance@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org"
  • Next message: Jim C. Nasby: "Re: How does disk caching work?"

    Relevant Pages

    • Re: read vs. mmap (or io vs. page faults)
      ... :with the possible madvise) provides the kernel with more information ... L2 cache, and that is precisely what is happening when you readinto ... a fixed buffer in a loop in a C program... ... and 3 GBytes/sec uncached to main memory. ...
      (freebsd-current)
    • Re: read vs. mmap (or io vs. page faults)
      ... :with the possible madvise) provides the kernel with more information ... L2 cache, and that is precisely what is happening when you readinto ... a fixed buffer in a loop in a C program... ... and 3 GBytes/sec uncached to main memory. ...
      (freebsd-questions)
    • Re: Recode to Play MP3?
      ... memory page 256 times, multiplied by the value in memory location $08. ... Your code is "perfect" for the cache, since the first page move "warms ... Since the write buffer can handle one byte every 8 cycles ... is not common in 6502 code, particularly loops. ...
      (comp.sys.apple2)
    • Re: Cache Coherency
      ... Don Burn (MVP, Windows DDK) ... memory buffer at PCI BAR4 which is marked as cacheable to the OS. ... Once the OS is done transferring data out of the buffer, ... internal cache with the new data since I can see no way that it can be ...
      (microsoft.public.development.device.drivers)
    • Re: Discovering variable types...
      ... >- but I suppose MS expect us to use wrappers ... memory allocations for your variables from disk as well. ... >They most certainly are of fixed size, changing the size of a String ... >>me to keep buffer size and current postion right in the memory block. ...
      (comp.lang.pascal.delphi.misc)