Re: how to flush out cache.?

From: Stephan Uphoff (ups_at_tree.com)
Date: 04/23/04

  • Next message: Andre Oppermann: "Re: events when (de)associating or when cable is (un)plugged" 
    To: Matthew Dillon <dillon@apollo.backplane.com>
Date: Thu, 22 Apr 2004 21:03:44 -0400

    Matthew Dillon wrote:
    > :Yes - but FreeBSD then calls vm_object_page_remove to remove the pages
    > :from the vnode object. (vm_object_sync for 5.x or vm_map_clean for 4.x )
    > < ... SNIP ...>
    >
    > I don't quite see that. Could you point out the code in question?
    > (And, of course, a test program would tell us for sure whether that
    > hole exists). There are two different things being removed... the
    > page table entries are removed from pmap via the vm_map for the process,
    > and the pages in the underlying object are being cleaned.
    >
    > Just removing pages from a pmap will not destroy the underlying pages.
    >

    For 4.X
            msync() -> vm_map_clean() -> vm_object_page_remove()

    4.X vm_map_clean()
            ....
            if (object && invalidate &&
                       ((object->type == OBJT_VNODE) ||
                        (object->type == OBJT_DEVICE))) {
                            vm_object_reference(object);
                            vm_object_page_remove(object,
                                OFF_TO_IDX(offset),
                                OFF_TO_IDX(offset + size + PAGE_MASK),
                                FALSE);
                            vm_object_deallocate(object);
                    }
             ...

    For 5.X (current)
            msync() -> vm_map_sync() -> vm_object_sync() -> vm_object_page_remove()

    vm_object_page_remove() in 4.X and 5.X calls vm_page_free() for pages that are
    not wired and zeroes p->valid for wired pages.

    Unfortunately this does not play well together with the buffer layer
    (vfs_bio.c).
    The layer caches vmio buffers and as such will keep the pages wired.
    On a cache hit it does not check if the pages of the vmio buffer are still
    valid. ( See test program "timetravel" why I consider this a bug)

    This means that a normal file read() or write() access using these buffers
    will act on pages marked invalid.

    However accesses through the mmap interface will encounter the invalid pages
    and force a reload.

    I wrote a few (throw away) test programs attached below.

    uncache.c: mmap()s a file and calls msync(...MS_INVALIDATE..)

    pagetouch.c: mmap()s a file and reads the first byte from every page to force
                   the pages to load/reload.
                   (and prints out the char sum of all the bytes)

    timetravel.c: shows an example on how the current vmio/msync interaction
                   can cause interesting problems.

    Try the following:
            
            dd if=/dev/zero of=testfile bs=8k count=32768
            time ./pagetouch testfile
            time ./pagetouch testfile
            ./uncache testfile
            time ./pagetouch testfile
            time ./pagetouch testfile

    Run "./timetravel testfile2" and look at the source to see some problems
    caused
    by the vmio/msync interaction bug.

    For the original problem the sequence (./uncache file ; ./pagetouch file)
    reloads all file data from stable storage.

    All programs were tested on 5.2.1

            Stephan

    PS: Postponing send-pr for a few days.

    _______________________________________________
    freebsd-hackers@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
    To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"

  • Next message: Andre Oppermann: "Re: events when (de)associating or when cable is (un)plugged"