Re: What is the PREEMPTION option good for?

:...
:the client. The difference is entirely due to dead time somewhere in
:nfs. Unfortunately, turning on PREEMPTION and IPI_PREEMPTION didn't
:recover all the lost performance. This is despite the ~current kernel
:having slightly lower latency for flood pings and similar optimizations
:for nfs that reduce the RPC count by a factor of 4 and the ping latency
:by a factor of 2.

The single biggest NFS client performance issue I have encountered
in an environment where most of the data can be cached from earlier
runs is with negative name lookups. Due the large number of -I
options used in builds, the include search path is fairly long and
this usually results in a large number of negative lookups, all of
which introduce synchronous dead times while the stat() or open()
waits for the over-the-wire transaction to complete.

The #1 solution is to cache negative namecache hits for NFS clients.
You don't have to cache them for long... just 3 seconds is usually
enough to remove most of the dead time. Also make sure your access
cache timeout is something reasonable.

It is possible to reduce the number of over-the-wire transactions to
zero but it requires seriously nerfing the access and negative cache
timeouts. It isn't usually worth doing.

Here are some test results:

make buildkernel, /usr/src mounted via NFS, 10 second access cache
timeout, multiple runs to pre-cache data and tcpdump used to verify
that only access RPCs were being sent over the wire for all tests.
(on DragonFly):

No negative cache - 440 seconds real
3 second neg cache timeout - 411 seconds real
10 second neg cache timeout - 410 seconds real (6% improvement)
30 second neg cache timeout - 409 seconds real

-Matt
Matthew Dillon
<dillon@xxxxxxxxxxxxx>
_______________________________________________
freebsd-arch@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: What is the PREEMPTION option good for?
    ... :for nfs that reduce the RPC count by a factor of 4 and the ping latency ... haven't figured the correct cache timeouts to use with it. ... hits tend to prevent RPCs until the cache times out. ... enough to remove most of the dead time. ...
    (freebsd-arch)
  • Re: Caching NFS Data
    ... >> errors about the NFS cache getting corrupted and needing to be checked. ... We've used it with success from Solaris 2.5.x I ... it was cachefs that Sun ships with Solaris. ...
    (comp.sys.sun.admin)
  • Re: How to manage shared persistent local caching (FS-Cache) with NFS?
    ... Imagine that the administrator requests a mount that uses part of a cache. ... NFS currently builds a key to the ... Fsids are supposed to be stable over server reboots. ...
    (Linux-Kernel)
  • Re: Security issues with local filesystem caching
    ... some background daemon so that they have no effect on NFS performance. ... on a cache miss, but it's not as simple as you make it out to be. ... readpages, unless you want to special-case the first readpages - in which case ... VM may have discarded them on memory pressure). ...
    (Linux-Kernel)
  • Re: [RFC][patch 0/2] mm: remove PageReserved
    ... > but whilst it's much faster than a disk, it is severely restricted in size ... Did you just suggest that 16 TB/address_space is too small to cache NFS pages? ... it would then unlock the page ...
    (Linux-Kernel)