RE: interrupt latency and driver locking

From: John Polstra (jdp_at_polstra.com)
Date: 09/20/03

  • Next message: Luigi Rizzo: "Re: interrupt latency and driver locking" 
    Date: Sat, 20 Sep 2003 09:48:20 -0700 (PDT)
To: Sam Leffler <sam@errno.com>

    On 19-Sep-2003 Sam Leffler wrote:
    > I enabled MUTEX_PROFILING on a fast machine I've got setup as a
    > firewall/router. The machine had an fxp device on one side and an em on
    > the other. I then ran a bunch of netperf tests through the machine.
    > Unfortunately for the moment I'm stuck with 100baseT on the fxp side of the
    > machine so my tests were all limited by the media. But one thing stuck out:
    [...]
    > Note that the fxp driver holds its driver lock for an average 112 us in one
    > spot. This turns out to be in fxp_tick and the likely culprit is the call
    > to mii_tick which is done with the lock held.

    Agreed. The MII calls usually have lots of DELAY() calls in them.

    > I'm not sure if it's simple to move MII operations outside of the driver
    > lock but it might be worth investigating.

    It seems like it ought to be possible to at least lock at a finer
    grain for those calls. The driver lock is held because the MII
    routines call back into the driver and use it to do the bit-twiddling
    necessary to shift data into and out of the PHY registers via the
    PHY's serial interface. That's normally pretty independent of
    anything else that's going on in the chip, so it could probably be
    done under a separate lock.

    Another idea would be to substitute something else for DELAY() calls
    of more than a few microseconds. I am not very up-to-date on -current
    these days, but as far as I know DELAY() is still a spin-wait. If
    the delay is for more than a few microseconds it would probably be
    better to switch to a different runnable thread and come back later.
    In fact, CPU speeds may have reached the point where that is always
    the right thing to do.

    Finally, the entire mii_tick() nonsense could be thrown out for many
    modern network adapters. Both the Broadcom and Intel gigabit chips
    are capable of interrupting on interesting link events, so there is no
    need at all to poll the PHY every second. That doesn't appear to be
    true of the fxp devices, though. They need to be polled. Still, you
    can avoid the expensive PHY reads and writes most of the time. For
    example, if you have received any packets on the interface in the past
    second, you can assume that link is good and never touch the PHY at
    all.

    John

    _______________________________________________
    freebsd-arch@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-arch
    To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"

  • Next message: Luigi Rizzo: "Re: interrupt latency and driver locking"