Re: packet drop with intel gigabit / marwell gigabit
- From: "Gary Thorpe" <gthorpe@xxxxxxxxxxxxx>
- Date: Tue, 21 Mar 2006 13:42:35 -0500
Jin Guojun [VFFS] wrote:
OxY wrote:
----- Original Message ----- From: "Jin Guojun [VFFS]" <g_jin@xxxxxxx>
To: "OxY" <oxy@xxxxxxxx>
Cc: <freebsd-performance@xxxxxxxxxxx>
Sent: Monday, March 20, 2006 4:05 AM
Subject: Re: packet drop with intel gigabit / marwell gigabit
....
First let's clear the notation -- Is 30MB/s (MBytes/s) = 240Mb/s
(Mbit/s) or MB/s means Mbits/s
If MB/s is MBytes/s and you also write this amount data to a disk,
plus other traffic on fxp0 to disk too,
then your problem may be bonded by memory bandwidth because CPU
utilization is low:
(240 + 24~32) x 2 is about 535 Mbit/s (some chipset/motherboard
has low memory BW for AMD)
If this is true, then this no thing you can tune. What does the
chipset (Motherboard) this machine have?
30MB/s is Megabytes/sec, currently i have 18-20MB/s peak and 15MB/s
avg.
it's not 535Mbit/s, because i only download it to my machine, no
upload.
disks are different from apache disks, these disks have own
controller in one pci slot.
the packet drop is 5-7% at 200Mbit iperf test, 100Mbit drop is
around zero.
i have <ASUS A7V8X> on motherboard which has VIA KT400 northbridge
http://uk.asus.com/products4.aspx?modelmenu=2&model=226&l1=3&l2=13&l3=62
Yes, this is one of problem chipset. I bought one about 3 years ago.
After one day testing, I returned it for changing a A7V600 (VIA KT600
chipset),
which is 30% more memory bandwidth than KT400. A7V600 can only
receive max
604 Mb/s TCP, so You can imagine what the KT400 can do :-)
I do not have a record (because it is too bad), but taking minimum
25% off,
it probably about 420-430 Mb/s (50MB/s). Now you can do the math when
the
machine also writing data to a disk (assume disk a fast enough). I
would expect
2/3 of 430 Mb/s, which is about 280~290 Mb/s (35 MB/s).
If you experiment these numbers, you are at there. No improvement you
can make
further.
i have doubts, because when i have 3-4MB/s traffic on fxp0 then em0 peak
is 18MB/s, but when fxp0 is almost idle, have 500kB/s traffic, then
em0 can only
do 20MB/s..
Since you did not get anything better than 35MB/s, then, what is your
doubt --
the maximum I/O A7V8X can do?
The 35 MB/s is the theoretical ceiling based on 2100+ CPU. 2000+ will be
slower.
In previous email, you mentioned you had 240 Mb/s (30 MB/s) on em0 with
some
traffic on fxp0, it is pretty much close to your hardware physical
limitation.
I thought all modern NICs used bus mastering DMA i.e. not dependent on CPU for data transfers? In addition, the available memory bandwidth for modern CPU's/systems is well over 100 MB/s. DDR400 is 400 MB/s (megabytes per second). Bus mastering DMA will be limited by the memory or IO bus bandwidth primarily. The system bus bandwidth cannot be the problem either: his motherboard's lowest front side bus speed is 200 MHz * 64-bit width = 1.6 GB/s (gigabytes per second) of peak system bus bandwidth.
The limitation of 32-bit/33 MHz PCI is 133 MB/s (again, megabytes not bits) maximum. Gigabit ethernet requires 125 MB/s (not Mb/s) maximum bandwidth: 32/33 PCI has enough for bursts but bus contention with disk bandwidth will reduce the sustained bandwidth. The motherboard in question has an option for integrated gigabit LAN which may bypass the shared PCI bus altogether (or it might not).
Anyway, the original problem was packet loss and not bandwidth. His CPU is mostly idle, so that cannot be the reason for packet loss. If 32/33 PCI can sustain 133 MB/s then it cannot be a problem because he needs
less than this. If it cannot, then packets will arrive too fast from the network before they can be moved from the board into memory and would cause the packet loss. Otherwise, his system is capable of achieving what he wants in theory and the suboptimal behavior may be due to hardware (e.g. PCI bus bandwidth not being able to reach 133 MB/s sustained) or software limitations (e.g inefficient operating system).
Forget drop in this figure, because this demonstrated how much hardware
can do,
rather than lossless transmission.
Once you have determined the ceiling, you need to keep a margin for
lossless Tx.
for other overhead, such as context switch, etc.
20 MB/s is not good enough for this board, you may expect 28-30 MB/s with
fine tuning. Unless you will be happy with 28 MB/s, it does not make
sense to
waste time to try to bump I/O above 30 MB/s for your application if you
have
another motherboard.
Again, this motherboard is designed for entertainment boxes not for network
I/O based applications.
-Jin
_______________________________________________
freebsd-performance@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to
"freebsd-performance-unsubscribe@xxxxxxxxxxx"
_______________________________________________
freebsd-performance@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@xxxxxxxxxxx"
- Follow-Ups:
- Re: packet drop with intel gigabit / marwell gigabit
- From: Jin Guojun [VFFS]
- Re: packet drop with intel gigabit / marwell gigabit
- Prev by Date: Re: packet drop with intel gigabit / marwell gigabit
- Next by Date: Re: packet drop with intel gigabit / marwell gigabit
- Previous by thread: Re: packet drop with intel gigabit / marwell gigabit
- Next by thread: Re: packet drop with intel gigabit / marwell gigabit
- Index(es):
Relevant Pages
|
|
