Re: Poor NFS server performance in 6.0 with SMP and mpsafenet=1

From: Jeremie Le Hen (jeremie_at_le-hen.org)
Date: 11/07/05

  • Next message: Robert Watson: "Re: RELENG_6 Kernel doesn't build due to DEVFS"
    Date: Mon, 7 Nov 2005 15:19:11 +0100
    To: Gavin Atkinson <gavin.atkinson@ury.york.ac.uk>
    
    

    Hi, Gavin, Martin,

    > Certainly. For heavily loaded machines, polling seems to significantly
    > improve performance, but in my experience on lightly loaded machines it
    > seems to increase latency on response to packets. For the particular
    > workloads it will be used for, (serving web content and databases), the
    > latency can be noticeable.
    >
    > IMHO, polling is great for lower-spec boxes or highly loaded machines,
    > but can hinder performance when the machine is lightly loaded. I have a
    > few machines which run with polling on, and for those usage patterns it
    > really helps. I just don't believe this is one of those cases.

    Just FYI, em(4) has powerful thresholds managements concerning what
    Intel calls "interrupt moderation". This is plainly described in
    this document (quite short to read) [1]. This looks like software
    polling except this is managed by the network adapter itself.

    Basically, em(4) network adapters have an "absolute timer" which
    achieves, IIUC, the same goal as our software polling : when the timer
    reaches zero, the adapter generates an interrupt. But those adapters
    also have a "packet timer", and this is very interesting and just
    impossible to emulate this behaviour with software polling. A packet
    timer is shorter than an absolute timer in order to decrease latency :
    when a packet is received the packet timer countdown begins. If
    another packet is received before it reaches zero, it is reset but
    goes on couting down... and so on, until one of the absolute timer or
    the packet timer timeouts. (I hope I'm clear enough).

    This allows to have the benefits of polling (limiting the interrupt
    rate) without having the latency introduced by software polling.

    AFAIK, this is tunable through these sysctls :
    % obiwan:gogo# sysctl dev.em.0 | grep delay
    % dev.em.0.rx_int_delay: 0
    % dev.em.0.tx_int_delay: 66
    % dev.em.0.rx_abs_int_delay: 66
    % dev.em.0.tx_abs_int_delay: 66

    Note that the packet timer is useless here.

    BTW, given the above feature, is there any benefits to use software
    polling of em(4) chips ?

    Best regards,

    [1] http://www.intel.com/design/network/applnots/ap450.pdf

    -- 
    Jeremie Le Hen
    < jeremie at le-hen dot org >< ttz at chchile dot org >
    _______________________________________________
    freebsd-current@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-current
    To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
    

  • Next message: Robert Watson: "Re: RELENG_6 Kernel doesn't build due to DEVFS"

    Relevant Pages

    • Re: One-shot high-resolution POSIX timer periodically late
      ... small bursts due to network jitter (typical average rate of 1000 packets ... Just before I re-send a packet, I arm a one-shot timer in order to ... Linux Plug and Play Support v0.97 Adam Belay ...
      (Linux-Kernel)
    • Re: Polling tuning and performance
      ... What I'm aiming for, of course, is zero packet loss. ... network traffic and that only by polling). ... so with most interrupts avoided by using polling it has little effect. ...
      (freebsd-performance)
    • Re: [antinvidia@gmail.com: some questions about bge(4)]
      ... known bugs cause brgphy_serviceto often lose a packet. ... if the system can't actually poll at 1/HZ. ... since at least 1500 packets of buffering would be needed for polling ... Polling in idle can work better if the system is actually idle. ...
      (freebsd-net)
    • Re: Ping & Physics
      ... As I'm sure you know ping retuns the time taken for the packet ... be determined from ping latency values, ... considering only the bit rate is 1.5 ms for a round trip. ...
      (comp.unix.questions)
    • Re: Polling For 100 mbps Connections? (Was Re: Freebsd Theme Song)
      ... Polling For 100 mbps Connections? ... from the network into the ethernet receiver. ... It takes a certain amount of time to get the packet out of ... not in the ethernet driver code. ...
      (freebsd-questions)