Re: how to saturate 100Mbit

From: Luigi Rizzo (rizzo_at_icir.org)
Date: 12/14/03

  • Next message: Nate Grey: "Re: Fwd: 5.2-RC + ipfw"
    Date: Sun, 14 Dec 2003 03:10:42 -0800
    To: Eugene Grosbein <eugen@www.svzserv.kemerovo.su>
    
    

    On Sun, Dec 14, 2003 at 11:29:07AM +0700, Eugene Grosbein wrote:
    > On Sat, Dec 13, 2003 at 10:17:06AM -0800, Luigi Rizzo wrote:
    >
    > > the fxp has a problem which does not allow it to go above 103/110/120kpps
    > > depending on which descriptor model you use, no matter how fast
    > > the CPU is.
    >
    > Can you explain the problem, please?

    Sorry if i sound a bit vague but did these tests a couple of years ago,
    and i do not have access to the fxp specs or contacts with people
    who designed the chip so i cannot say what is exactly the problem.

    The problem is in the hardware, not in the driver. Apparently the chip
    wastes a lot of time (couple of microseconds if i remember well)
    between reading the descriptor and then transmitting the data.
    This extra delay is in parallel with some other chip activity, so
    for larger frames (say 256 bytes) it is completely masked and packets
    are transmitted at the nominal rate, but smaller packet are sent as if
    the interframe gap were larger than the standard.

    At the time i did a lot of experiments to make sure that the NIC
    was not stalled by the cpu not supplying frames in time, etc., and
    all tests confirmed the above diagnosis.
    BTW it is not something that has to do with flow control frames,
    because enabling/disabling them does not change the throughput (and you
    can see the frames when they are not disabled).

    Changing the way descriptor are used (there are two different
    formats, one has something like the data right after the descriptor)
    helped increase the performance to 120kpps, whereas the standard
    freebsd driver is stuck at something between 103 and 110kpps.

    Not that it matters terribly, just useful to know when you have
    to push small frames at line rate and you find you can't with those
    cards. There are others (e.g. intel/dec 2114x, 3com's xl, possibly
    others, plus probably all of the gbit units when used at 100Mbit)
    which work fine at line rate with 64-byte frames.

    > > Even not using any special kernel modules, a simple loop over
    > > a sendto() on a udp socket can achieve around 500kpps on a 2.4GHz
    > > box (em or bge). With some tricks and a sufficiently fast PCI
    > > bus you can reach some 750kpps but then it really depends
    > > on how fast is your PCI bus.
    A
    >
    > 100*1024*1024/8/1500=8738.1(3)
    >
    > It seems one does not need hundred of thousand pps to achive 100Mbps.

    certainly not, but because most of the time the overhead is mostly per
    packet, if you want to run performance tests you care about
    packets, not data rate.

            cheers
            luigi
    _______________________________________________
    freebsd-net@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-net
    To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"


  • Next message: Nate Grey: "Re: Fwd: 5.2-RC + ipfw"

    Relevant Pages

    • RE: Problems with BCE network adapter (Dell PE2950)
      ... There's already some debug code to watch for unusual size packets. ... passing an Ethernet frame up the stack with a length of -4. ... Is there anything unusual about your network setup like VLAN ... The network is running Jumbo frames. ...
      (freebsd-net)
    • Re: Statistics Extraction
      ... 546 confirmed frames sent succesfully ... 549 packets sent successfully ... packets dropped due to no receiver ...
      (comp.lang.perl.misc)
    • Re: atheros driver under high load, panics and even more freezes
      ... 19308912 data frames received ... 62583 tx failed 'cuz too many retries ... 27129 tx stopped 'cuz no xmit buffer ... 10000 packets transmitted, 10000 packets received, 0% packet loss ...
      (freebsd-stable)
    • Re: DirectShow DirectSound microphone low-latency
      ... For video you can adjust the timestamps and it will speed up or slow down ... frames at the decoder. ... The main problem is due to dropping video packets. ... until we run out of extra buffers. ...
      (microsoft.public.win32.programmer.directx.audio)
    • File server getting DoSd by forged packets???
      ... Sorry for the length but this is a real problem to me! ... The pattern of these packets is very distinct, ... these frames is exactly 1500. ... This capture was taken Monday 9/16/2002. ...
      (Security-Basics)