Throughput problems in RELENG_7

I'm seeing a problem where a much faster quad-core host running RELENG_7
serves many fewer netrate/http requests per second (175/sec) than an
old, busy, UP 6.0 host (828/sec). The problem seems to be related to
latency and connection setup, as it shows up dramatically over a link
with 50-60 ms latency. Can you help?

"Gandalf" is the problem RELENG_7 host connected via em0 on a 100baseTX
full-duplex switch port.
"Boromir" is a 6.0 host for reference, connected via bge0 to another
100baseTX full-duplex port on the same switch.
"Theoden" is a 6.2 host remote (50-60 ms away) from gandalf and boromir.

Running tools/netrate/http against tools/netrate/httpd on gandalf and
boromir, serving a document with 35 bytes of content, I get this
(details including tcpdumps following my sig):

dest dest
source boromir gandalf
------ ------- -------
theoden 828 175
boromir - 18092
gandalf 11493 -

The problem occurs using ab as well, but is much less pronounced when -k
is used (enabling keepalive and presumably HTTP pipelining). This makes
me wonder if the problem might have something to do with connection
setups.

I've verified that the switch port and the interface are configured
correctly (100baseTX full-duplex). I've also tried toggling polling and
the following sysctls, with no significant change:

net.inet.tcp.tso
net.inet.tcp.newreno
net.inet.tcp.sack.enable
net.inet.tcp.inflight.enable
net.inet.tcp.syncookies_only
net.inet.tcp.syncookies

What else can I try?

Thanks for any help you can give.

Matt




theoden-gandalf
===============

theoden:~$ /usr/obj/usr/src/tools/tools/netrate/http/http -s 2
216.193.240.4
1280 /short.html
187 transfers/second
0 errors/second

tcpdump:
http://gandalf.vpop.net/~mreimer/theoden-gandalf-netrate.dump.gz

---

theoden:~$ /usr/obj/usr/src/tools/tools/netrate/http/http -n 100 -s 2
216.193.240.4 1280 /short.html
175 transfers/second
0 errors/second

tcpdump:
http://gandalf.vpop.net/~mreimer/theoden-gandalf-netrate-n100.dump.gz



theoden-boromir
===============

theoden:~$ /usr/obj/usr/src/tools/tools/netrate/http/http -s 2
216.193.240.2
1280 /short.html
272 transfers/second
0 errors/second

tcpdump:
http://boromir.vpop.net/~mreimer/theoden-boromir-netrate.dump.gz

---

theoden:~$ /usr/obj/usr/src/tools/tools/netrate/http/http -n 100 -s 2
216.193.240.2 1280 /short.html
828 transfers/second
0 errors/second

tcpdump:
http://boromir.vpop.net/~mreimer/theoden-boromir-netrate-n100.dump.gz


boromir-gandalf
===============

boromir: $ /usr/obj/usr/src/tools/tools/netrate/http/http -n 100 -s 2
216.193.240.4 1280 /short.html
18092 transfers/second
0 errors/second

gandalf-boromir
===============

gandalf: $ /usr/obj/usr/src/tools/tools/netrate/http/http -n 100 -s 2
216.193.240.2 1280 /short.html
11493 transfers/second
0 errors/second




boromir sysctl and ifconfig:

http://boromir.vpop.net/~mreimer/boromir.sysctl.gz


# ifconfig bge0
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=1a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
inet6 fe80::2e0:81ff:fe60:3925%bge0 prefixlen 64 scopeid 0x1
inet 216.193.240.33 netmask 0xffffffff broadcast 216.193.240.33
inet 216.193.240.34 netmask 0xffffffff broadcast 216.193.240.34
inet 216.193.240.35 netmask 0xffffffff broadcast 216.193.240.35
inet 216.193.240.36 netmask 0xffffffff broadcast 216.193.240.36
inet 216.193.240.37 netmask 0xffffffff broadcast 216.193.240.37
inet 216.193.240.17 netmask 0xffffffff broadcast 216.193.240.17
inet 216.193.240.38 netmask 0xffffffff broadcast 216.193.240.38
inet 216.193.240.39 netmask 0xffffffff broadcast 216.193.240.39
inet 216.193.240.2 netmask 0xfffff800 broadcast 216.193.247.255
inet 216.193.240.40 netmask 0xffffffff broadcast 216.193.240.40
ether 00:e0:81:60:39:25
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active

gandalf sysctl and ifconfig:
http://gandalf.vpop.net/~mreimer/gandalf.sysctl.gz

# ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
1500
options=1cb<RXCSUM,TXCSUM,VLAN_MTU,POLLING,VLAN_HWCSUM,TSO4>
ether 00:30:48:33:96:a4
inet 216.193.240.4 netmask 0xfffff800 broadcast 216.193.247.255
inet 216.193.240.65 netmask 0xffffffff broadcast 216.193.240.65
inet 216.193.240.66 netmask 0xffffffff broadcast 216.193.240.66
inet 216.193.240.67 netmask 0xffffffff broadcast 216.193.240.67
inet 216.193.240.68 netmask 0xffffffff broadcast 216.193.240.68
media: Ethernet 100baseTX <full-duplex>
status: active


_______________________________________________
freebsd-net@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@xxxxxxxxxxx"

Relevant Pages