Re: Network sysctl tuning [was Re: ZFSKnownProblems - needs revision?]

On Wed, Apr 8, 2009 at 3:55 PM, Antony Mawer <fbsd-stable@xxxxxxxxx> wrote:
Freddie Cash wrote:
...
We've also heavily modified /etc/sysctl.conf and upped a bunch of the
network-related sysctls.  Doing so increased our SSH throughput from ~30
Mbits/sec across all connections to over 90 Mbits/sec per SSH connection..

Are you able to share any of these with the list? It would be useful to
compare as a lot of these tunings people do individually and it would be
good to allow others to test in their environments to see if they help, as
well as potentially adding them to the tuning man-page.

They're all taken from the HPN-SSH website and various google searches
related to HPN-enabled OpenSSH.

I don't know exactly what all the different, individual sysctls do,
nor whether this is the most optimal setup, but here's the sysctl.conf
that we use. This is on 2 systems using a quad-port gigabit NIC where
the top two ports are connected via lagg(4) and the bottom two ports
are connected via lagg(4), with the two laggX interfaces on separate
networks.

I did a bunch of scp/sftp transfers of 100 MB files filled with random
data pulled from /dev/random between these two boxes tweaking the
options one at a time, but didn't do too much in the way of
scientific/empirical measurements and comparisons beyond the
throughput data that scp/sftp shows.

If there are any glaring errors, gotchas, or "why would you ever do
that"s, let me know. :)

# General network settings
net.isr.direct=1 # Whether to enable Direct
Dispatch for netisr


# IP options
net.inet.ip.forwarding=0 # Whether to enable packet
forwarding for NAT/routing
net.inet.ip.process_options=0 # Disable processing of IP
options (nothing uses this field)
net.inet.ip.random_id=1 # Randomise the IP header ID number
net.inet.ip.redirect=0 # Whether to allow redirect packets
#net.inet.ip.stealth=0 # Whether to appear in traceroute output


# ICMP options
net.inet.icmp.icmplim=200 # Limit ICMP packets to this
many per second
net.inet.icmp.drop_redirect=1 # Drop ICMP redirect packets
net.inet.icmp.log_redirect=0 # Don't log ICMP redirect packets


# TCP options
net.inet.tcp.blackhole=1 # Drop packets destined to unused ports
net.inet.tcp.inflight.enable=0 # Use automatic TCP window-scaling
net.inet.tcp.log_in_vain=0 # Don't log the blackholed packets
net.inet.tcp.path_mtu_discovery=1 # Use ICMP type 3 to find the MTU to use
net.inet.tcp.recvbuf_max=16777216 # The max size of the receive
buffer (16 MB)
net.inet.tcp.recvspace=131072 # The initial size in bytes of
the receive buffer
net.inet.tcp.sack.enable=1 # Enable Selective ACKs
net.inet.tcp.sendbuf_max=16777216 # The max size of the send buffer
net.inet.tcp.sendspace=131072 # The initial size in bytes of
the send buffer
net.inet.tcp.syncookies=1 # Enable SYN cookie protection
net.inet.tcp.rfc1323=1 # Enable RFC1323 extensions
(TCP window scaling)


# UDP options
net.inet.udp.blackhole=1 # Drop packets destined to unused ports
net.inet.udp.checksum=1 # Enable UDP checksums
net.inet.udp.log_in_vain=0 # Don't log the blackholed packets
net.inet.udp.recvspace=65536 # Size in bytes of the receive buffer


# Debug options
debug.minidump=1 # Disable the small kernel
core dump (only mem in use)
debug.mpsafevfs=1 # Enable threaded VFS subsystem


# Kernel options
kern.coredump=0 # Disable kernel core dumps
kern.ipc.maxsockbuf=4194304 # Set the max size of the
socket buffers (4 MB)
kern.ipc.somaxconn=1024 # Expand the IP listen queue
kern.maxvnodes=250000 # Bump up the max number of vnodes


# PCI bus options
hw.pci.enable_msix=1 # Enable Message Signalled
Interrupts - Extended
hw.pci.enable_msi=1 # Enable Message Signalled Interrupts
hw.pci.enable_io_modes=1 # Enable alternate I/O access modes

--
Freddie Cash
fjwcash@xxxxxxxxx
_______________________________________________
freebsd-stable@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: verizon FIOS +linux
    ... I got an offer in the mail from verizon for a fiber optics internet ... Mbits/sec down, 2 Mbits/sec up ... router and activated the service. ... I know that they do not block all connections to ports <1024, ...
    (comp.os.linux.misc)
  • Re: Traffic control: throttling downloads
    ... You might consider limiting or otherwise disabling Linux's autotuning ... net.ipv4.tcp_rmem and have it apply to all connections. ... One thing I did here long ago was to limit the size of the outgoing buffer ... I can leave enough bandwidth for other uses while, for example, downloading ...
    (comp.os.linux.networking)
  • RE: NT4 and 2000 Server do not apear in the My Network Places
    ... the Server service does not bind to the ... list must also have file and printer sharing turned on. ... Internet Connections", ... Networks" is in the list of components for the connection. ...
    (microsoft.public.win2000.networking)
  • RE: Win2k network cant browse
    ... the Server service does not bind to the ... list must also have file and printer sharing turned on. ... Internet Connections", ... Networks" is in the list of components for the connection. ...
    (microsoft.public.win2000.networking)