Network performance issues when writing to disk (5.2.1-RELEASE)

From: Erik Rothwell (erik.rothwell_at_realityengine.ca)
Date: 08/11/04

  • Next message: David Burns: "Re: Network performance issues when writing to disk (5.2.1-RELEASE)" 
    Date: Tue, 10 Aug 2004 20:10:22 -0400
To: <freebsd-performance@freebsd.org>

    I have a workgroup server running FreeBSD 5.2.1 that is experiencing some
    serious network performance problems over very minimal load.

    The server has two miibus-based NICs: a WAN link via dc1 and a switched LAN
    serving ~7 clients attached via dc0.

    File transfers to the server (eg, scp, WebDAV, NFS, FTP) experience terrible
    throughput, starting off at about 2MB/s and dropping rapidly to ~200KB/s.

    At first, I suspected duplex mismatch or a misbehaving switch (unmanaged
    switch with all client interfaces set to autosense), but the problem can be
    replicated with the server connected directly to any of the clients with the
    media at both ends explicitly set. netstat -ni indicates no collisions but a
    few I/Oerrs during transfers.

    It appears from testing that the interface itself responds intermittently
    under this load (during transfers, the switch reports the port is
    intermittently inactive -- ifconfig on the server will also sometimes report
    "status: inactive") and packet loss approaches 75%. Occasionally, "dc0: TX
    underrun -- increasing TX threshold" would crop up in dmesg, but that's not
    usually noteworthy.

    I replaced the NIC, yet the problem curiously persists.

    After further testing, it seems the problem is isolated only to network file
    transfers that actually write to disk.

    For instance, using netcat to pipe random junk to /dev/null at the server,
    throughput is good and the NIC behaves itself:

    # nc -l -p 2332 -vv | dd if=/dev/stdin of=/dev/null bs=1k
    listening on [any] 2332 ...
    connect to [192.168.5.1] from [192.168.5.3] 51295
     sent 0, rcvd 104857600
    94731+21879 records in
    94731+21879 records out
    104857600 bytes transferred in 11.897846 secs (8813158 bytes/sec)

    When writing to a file, however:

    # nc -l -p 2332 -vv | dd if=/dev/stdin of=/data/junk bs=1k
    listening on [any] 2332 ...
    connect to [192.168.5.1] from [192.168.5.3] 51296
     sent 0, rcvd 1130496
    1046+115 records in
    1046+115 records out
    1130496 bytes transferred in 32.888345 secs (34374 bytes/sec)

    Conversely, using netcat to pipe a 1GB file on the server to one of the
    clients was no problem:

    # nc -l -p 2332 -vv > /tmp/junk
    ...
    698018816 bytes transferred in 95.423028 secs (7314993 bytes/sec)

    Locally, disk performance isn't suffering unusually:

    # dd if=/data/junk of=/dev/null
    2246100+0 records in
    2246100+0 records out
    1150003200 bytes transferred in 97.113807 secs (11841809 bytes/sec)

    # dd if=/dev/zero of=/data/junk bs=1k count=1m
    1048576+0 records in
    1048576+0 records out
    1073741824 bytes transferred in 50.131779 secs (21418387 bytes/sec)

    After searching the lists and documentation, I'm still not sure what's
    causing the problem. Has anyone seen anything of this nature before? Any
    suggestions?

    Erik.

    _______________________________________________
    freebsd-performance@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-performance
    To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org"

  • Next message: David Burns: "Re: Network performance issues when writing to disk (5.2.1-RELEASE)"

    Relevant Pages

    • RE: SBS2k3 VPN Issue
      ... Your SBS server has only one Nic. ... VPN clients to the same IP as the LAN, the clients can RDP either Server or ... When the server assigns the VPN clients the different IP ... <between the internal and external nics, hence routing the request, or have ...
      (microsoft.public.windows.server.sbs)
    • Re: Win2K RRAS/VPN Help
      ... > if you try to configure different gateway settings on the NICs). ... > on the server itself. ... *.201 thru *.225 IPs for VPN clients. ... RRAS using *.200 and clients getting *.200+ IPs. ...
      (microsoft.public.win2000.ras_routing)
    • Re: SBS 2003 2NICs
      ... Some very good advice here on routing, IP's, disaster recovery etc. ... ISP in the UK in my opinion - I've got 3 clients using them and incompetent ... If your server ... the external and internal NICs need to be on ...
      (microsoft.public.windows.server.sbs)
    • Re: Problem using Remoting with 2 NICs
      ... The server itself is currently running in a dual NIC configuration at ... I have clients connecting successfully on NIC 1, ... When both NICs are enabled. ...
      (microsoft.public.dotnet.framework.remoting)
    • Re: Help with Linksys router and Windows Small Business Server 2003
      ... You should be using 2 nics in the server. ... All clients have dynamic IP's ... > should connect them to the internet providing router. ...
      (microsoft.public.windowsxp.network_web)