Re: serious networking (em) performance (ggate and NFS) problem
From: Robert Watson (rwatson_at_freebsd.org)
Date: Thu, 18 Nov 2004 12:27:44 +0000 (GMT) To: Emanuel Strobl <Emanuel.Strobl@gmx.net>
On Wed, 17 Nov 2004, Emanuel Strobl wrote:
> I really love 5.3 in many ways but here're some unbelievable transfer
> rates, after I went out and bought a pair of Intel GigaBit Ethernet
> Cards to solve my performance problem (*laugh*):
I think the first thing you want to do is to try and determine whether the
problem is a link layer problem, network layer problem, or application
(file sharing) layer problem. Here's where I'd start looking:
(1) I'd first off check that there wasn't a serious interrupt problem on
the box, which is often triggered by ACPI problems. Get the box to be
as idle as possible, and then use vmstat -i or stat -vmstat to see if
anything is spewing interrupts.
(2) Confirm that your hardware is capable of the desired rates: typically
this involves looking at whether you have a decent card (most if_em
cards are decent), whether it's 32-bit or 64-bit PCI, and so on. For
unidirectional send on 32-bit PCI, be aware that it is not possible to
achieve gigabit performance because the PCI bus isn't fast enough, for
(3) Next, I'd use a tool like netperf (see ports collection) to establish
three characteristics: round trip latency from user space to user
space (UDP_RR), TCP throughput (TCP_STREAM), and large packet
throughput (UDP_STREAM). With decent boxes on 5.3, you should have no
trouble at all maxing out a single gig-e with if_em, assuming all is
working well hardware wise and there's no software problem specific to
(4) Note that router latency (and even switch latency) can have a
substantial impact on gigabit performance, even with no packet loss,
in part due to stuff like ethernet flow control. You may want to put
the two boxes back-to-back for testing purposes.
(5) Next, I'd measure CPU consumption on the end box -- in particular, use
top -S and systat -vmstat 1 to compare the idle condition of the
system and the system under load.
If you determine there is a link layer or IP layer problem, we can start
digging into things like the error statistics in the card, negotiation
issues, etc. If not, you want to move up the stack to try and
characterize where it is you're hitting the performance issue.
Robert N M Watson FreeBSD Core Team, TrustedBSD Projects
email@example.com Principal Research Scientist, McAfee Research
firstname.lastname@example.org mailing list
To unsubscribe, send any mail to "email@example.com"