High traffic NFS performance and availability problems

From: David Rice (drice_at_globat.com)
Date: 02/18/05

  • Next message: Claus Guttesen: "Re: High traffic NFS performance and availability problems" 
    To: freebsd-performance@freebsd.org
Date: Thu, 17 Feb 2005 16:36:10 -0800

    We are a web hosting company that runs exclusivly on FreeBSD. We are having
    storage availability and performance problems. All of our storage is exported
    via NFS to the client machines Any suggestions or advice will be greatly
    appreciated. We are willing to pay someone on a consulting basis to help us
    solve these problems. Please email me off list if you are a consultant.

    On the file server side we have:

    Dell PowerEdge 1750's
    Dell Perc4 RAID controller
    Dell Power Vault 220 storage shelf.
    (12) 146GB SCSI drives on one SCSI Bus with a hot spare (~1.3TB file system)
    GigaBit ethernet to the NFS client machines
    1GB RAM
    (2) 2.4 GHZ Xenon Processors
    FreeBSD 5.2.1 or FreBSD 5.3

    On the client side we have:

    Dell PowerEdge 1750's
    1GB RAM
    (2) 2.4 GHZ Xenon Processors
    GigaBit ethernet
    36GB SCSI root disk
    FreeBSD 4.9 to FreeBSD 4.11

    On the network side we have:
    Gigabit ethernet
    Foundry BigIron and NetIron swithes
    Cisco 6509 with Gigabit switch blades

    Typicly we have 7 client boxes mounting storage from a single file server.
    Each client box servers 1000 web sites and associate email. We have
    done the basic NFS tuning (ie: Read write size optimization and kernel tuning)

    The problems we are having is as follows.

    1. Slow perfomance during peek traffic periods
    2. Client boxes have high load averages and sometimes crashes due to slow NFS
    performance.
    3. File servers that randomly crash with "Fatal trap 12: page fault while in
    kernel mode"
    4. With soft updates enabled during FSCK the fileserver will freeze with all
    NFS processs in the "snaplck" state. We disabled soft updates because of
    this.

    I can provide and other details about our configuration if needed.

    David Rice
    drice@globat.com

    Thank You.

    _______________________________________________
    freebsd-performance@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-performance
    To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org"

  • Next message: Claus Guttesen: "Re: High traffic NFS performance and availability problems"

    Relevant Pages

    • Re: High traffic NFS performance and availability problems
      ... > storage availability and performance problems. ... > via NFS to the client machines Any suggestions or advice will be greatly ... > GigaBit ethernet to the NFS client machines ...
      (freebsd-performance)
    • Re: is NFS production-ready ?
      ... My goal is to provide NFS service to many FreeBSD clients sharing the ... NFS servers sharing its LUNs. ... implementation of NFS client ...
      (freebsd-net)
    • Re: Kerberos to file server
      ... between a netbooted client and a regular PC client. ... NFS doesn't really /do/ permissions, so the easiest (and probably least ... individually mount the NFS share from as400 on boot, and since the FreeBSD ... I have no idea if as400 would make a nice AFS server. ...
      (FreeBSD-Security)
    • Re: Cheap NAS using FreeBSD - practical considerations?
      ... > network-attached storage, serving files by NFS, with gigabit ethernet. ... freebsd is a fine choice for this application. ... vary depending on what you're running on the clients; ...
      (freebsd-questions)
    • Re: High traffic NFS performance and availability problems
      ... > We are a web hosting company that runs exclusivly on FreeBSD. ... > having storage availability and performance problems. ... flying at the local disk speed. ... My use of NFS is for administration and user ...
      (freebsd-performance)