interrupt latency and driver locking

From: Sam Leffler (sam_at_errno.com)
Date: 09/20/03

  • Next message: John Polstra: "RE: interrupt latency and driver locking" 
    Date: Fri, 19 Sep 2003 15:59:15 -0700
To: freebsd-arch@freebsd.org

    I enabled MUTEX_PROFILING on a fast machine I've got setup as a
    firewall/router. The machine had an fxp device on one side and an em on
    the other. I then ran a bunch of netperf tests through the machine.
    Unfortunately for the moment I'm stuck with 100baseT on the fxp side of the
    machine so my tests were all limited by the media. But one thing stuck out:

       max total count avg name
       282 786 3 262 kern/kern_exec.c:853 (vm page queue
    mute
       775 7450 33 225 kern/vfs_subr.c:3218 (mntvnode)

      1119 2789 15 185 kern/kern_fork.c:218 (Giant)

       222 2344 14 167 vm/uma_core.c:1186 (UMA lock)

       460 2972 19 156 kern/kern_exit.c:282 (vm page queue
    mute
       508 2415 19 127 kern/kern_exit.c:101 (Giant)

       194 1801 16 112 kern/kern_exec.c:261 (Giant)

       332 31113 277 112 dev/fxp/if_fxp.c:1798 (fxp0)

      2435 31848 366 87 kern/kern_sysctl.c:1210 (Giant)

       901 5306 66 80 vm/vm_map.c:1256 (Giant)

        77 77 1 77 kern/uipc_syscalls.c:472 (Giant)

       182 2901 39 74 vm/vm_kern.c:328 (system map)

       507 841090 12508 67 dev/random/yarrow.c:172 (random
    reseed)
      7005 68335 1136 60 kern/kern_intr.c:533 (Giant)

        66 964 16 60 vm/vm_map.c:2241 (system map)

        88 7471 139 53 dev/em/if_em.c:1531 (em0)

       881 8032 191 42 i386/i386/trap.c:997 (Giant)

       591 10568722 271157 38 net/netisr.c:215 (netisr lock)

        38 292 8 36 vm/vm_fault.c:997 (Giant)

    Note that the fxp driver holds its driver lock for an average 112 us in one
    spot. This turns out to be in fxp_tick and the likely culprit is the call
    to mii_tick which is done with the lock held. Since there's only one lock
    in the driver this means interrupts are blocked during this time which is
    likely to do bad things. (The em driver comparison has a similar
    bottleneck though it's <1/2 as long.)

    I'm not sure if it's simple to move MII operations outside of the driver
    lock but it might be worth investigating.

            Sam

    _______________________________________________
    freebsd-arch@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-arch
    To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"

  • Next message: John Polstra: "RE: interrupt latency and driver locking"

    Relevant Pages

    • Re: OS Question
      ... driver, should not distress you. ... I've made the unwarranted assumption that having multiple threads is ... To protect data in user mode ... Permanence (if apps all crash the lock state is retained because ...
      (microsoft.public.development.device.drivers)
    • Re: LOR with netisr changes
      ... ::> It's just a bug in the driver. ... the softc and is intended to be covered by the driver's softc lock. ... this is a hack to work around a recursion in net80211. ...
      (freebsd-current)
    • Re: [PATCH] Remove process freezer from suspend to RAM pathway
      ... My question referred to drivers trying to bind or unbind a device ... The thread trying to bind is holding a lock which is ... Spinning in the driver with the lock not held is impossible, ... would be easiest to jump directly into the freezer! ...
      (Linux-Kernel)
    • Re: [patch 00/15] clocksource / timekeeping rework V4 (resend V3 + bug fix)
      ... which lock already depends on the new lock. ... now at 1915353302473 nsecs ... # You can enable one or both FireWire driver stacks. ...
      (Linux-Kernel)
    • Re: 6.0-BETA2: taskqueue_drain for if_xl.c:2796
      ... let's go back to the detach case where you actually need a drain. ... guarantee that that won't be forever since it can ensure the task handler ... with the lock held the entire time and doesn't drop it. ... Also, in general the entry points to the driver (ifnet functions, ifmedia ...
      (freebsd-current)