Re: Device polling performance

TM4525_at_aol.com
Date: 09/27/04

  • Next message: Gordon Freeman: "Re: Bind 9.3.0 on FreeBSD 5.2.1" 
    Date: Mon, 27 Sep 2004 15:07:27 EDT
To: stheg_olloydson@yahoo.com

    In a message dated 9/25/04 4:24:07 PM Eastern Daylight Time,
    stheg_olloydson@yahoo.com writes:
    >The EVIDENCE is to the contrary, since it seems that a 2.4Ghz system
    >will be saturated when bridging ~250Kpps with device-polling enabled,
    >based on polling stats and userland benchmarking, even though the
    >system claims to be 100% idle. Interestingly, its about the same with
    >interrupt enabled.
    >
    >The POINT is that since there is no way to measure the performance,
    >you've got a bunch of guys who think they've figured something out
    >touting device-polling without having a clue what the performance
    >advantages (or consequences) are, so it might as well be black magic,
    >or snake oil, since you are as blind as a bat in your assessments.

    Hello,

    Please post your "polling stats and userland benchmarking" results. I
    would be very interested seeing them as I was thinking of moving to
    NICs that would benefit from polling. However, because you have
    "EVIDENCE ... to the contrary", I may hold off. On the other hand, you
    do go on to say "there is no way to measure the performance" and "you
    are as blind as a bat in your assessments", so also please post your
    test methodology. I need to make my decision on reliable, repeatable
    facts.
    Also, when you post, would you please wrap your lines to a shorter
    length? Not everyone on the list uses AOL Reader, like you.
    ------------------------------------------------------------------------------
    ---------------------------
    The "evidence" is a bit circumstantial in the absence of working tools, but
    here are some "observations". There's also an assumption that the knobs
    associated with polling work as expected.

    Test machine is a 2.4Ghz celeron box with dual Intel NICs (em driver)
    on a 32bit, 33Mhz bus, running FreeBSD 4.9. Now I realize that a 133Mhz,
    64bit bus is 8x faster and you certainly wouldnt use these NICs on a real
    network, but for the purpose of a control it doesnt matter, since both
    tests are on the same MB.

    Settings:

    HZ=1000
    each_burst=512
    max_burst=1000
    user_frac=variable

    RXdescriptors (receive ring size) = 512

    (Note that the burst never exceeded 100 at any time)

    I'm firing a controlled stream of 100K pps through the box (bridging). With
    only normal userland (idle) usage, the box happily goes about its way.
    Top shows 0-1.5% usage.

    I started a cpu intensive userland task (buildworld or something of the sort).
    The system started to lose packets with a user_frac setting of 78, which
    implies that the system requires about 22% of the cpu to successfully
    manage the task, assuming the knob works (it appears to). The same
    machine, with interrupts enabled, uses about 26%, according to "top".
    HOWEVER, setting hz back to 100, with interrupts enabled the usage
    went down under 25%. Given that, it can be argued that there is less
    than a 5% bonus for polling, which makes a lot more sense than what
    some of the kooks have been saying.

    Of course the point here wasn't to prove the difference, which Im still
    not sure of, but the evidence certainly is that "top" doesn't properly
    account for CPU usage in device_polling mode.. I'd expect a small
    bonus, but nothing earth-shattering, as the machine still has to do
    the same amount of work. Its not like the machine is really servicing
    an interrupt for every packet, since controllers have hold offs so they
    don't generate interrupts on top of each other, and multiple events
    are regularly handled with a single interrupt.

    Polling gives the appearance of a machine happily going about its
    business no matter how much traffic you throw at it, but what happens
    is that you lose packets when it becomes overmatched, which never
    happens on a system with interrupts enabled before it goes into livelock.
    While livelock isn't a good thing, if it only happens occasionally, at least
    you aren't losing packets. Additionally, with a HZ setting of 1000 you're
    also introducing quite a bit of latency:

    additional_latency = up to 1ms in receive ring + transmit time for burst-1
    frames.

    Increasing HZ further would reduce the latency, but adds more overhead,
    which slims the advantages and defeats the purpose of polling in the first
    place.

    The bottom line is that there isn't so much difference as to think that
    polling
    is going to save the day, but if you don't care about latency or losing
    packets
    it can be useful in allocating cpu cycles to user space, if thats your
    priority.

    TM
    _______________________________________________
    freebsd-questions@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-questions
    To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org"

  • Next message: Gordon Freeman: "Re: Bind 9.3.0 on FreeBSD 5.2.1"

    Relevant Pages