Re: Locking etc. (Long, boring, redundant, newbie questions)

On Wed, Mar 28, 2007 at 10:40:58AM +0100, Robert Watson wrote:

Spin locks are, FYI, slower than default mutexes. The reason is that they
have to do more work: they not only perform an atomic operation/memory
barrier to set the cross-CPU lock state, but they also have to disable
interrupts to synchronize with fast interrupt handlers. In general, you
are right: you should only use a spin mutex if you are running in a fast
handler, or synchronizing with a fast handler. The one general exception
is the scheduler itself, which must protect its data structures with spin
locks in order to implement sleeping primitives. As such, the scheduler
lock and various other low level locks (such as in turnstiles) are
implemented with spin locks and not default mutexes. Since default mutexes
spin adaptively, the reduced overhead of contention experienced with spin
locks (i.e., no scheduler overhead for simple contention cases) is also
experienced with default mutexes.

By the way, could the following observation of mine be related to
the high cost of spin mutexes?

While doing some measurements of the performance of our vlan driver
in a router, I found that with RELENG_6 the pps rate through the
router degraded considerably (by ~100Kpps) if its physical interfaces
used hardware vlan tagging. I attributed that to the overhead of
allocating and freeing a mbuf tag for each packet as it entered and
then left the router. I used hwpmc and pmcstat to see which kernel
functions took most time and found that critical_exit() made top 5
in the list of CPU time eaters if the network interfaces were using
hardware vlan tagging.

The machine was an UP amd64, but it ran FreeBSD/i386, with an UP
kernel.

As I can see from the code, critical_exit() may grab and later
release a spin mutex. I've got a hazy recollection that our kernel
memory allocator uses critical sections to protect its per-CPU
structures. That's why I suspect that the effect I observed may
have its roots in the behaviour of spin mutexes. Could it be so?

Thanks!

--
Yar
_______________________________________________
freebsd-hackers@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: Locking etc. (Long, boring, redundant, newbie questions)
    ... > Spin locks are, FYI, slower than default mutexes. ... > implemented with spin locks and not default mutexes. ... I used hwpmc and pmcstat to see which kernel ...
    (freebsd-hackers)
  • Re: [PATCH][RFC]: mutex: adaptive spin
    ... It's not the typical spin-mutex i think. ... long should we spin" question elegantly: we spin until the owner runs on a ... well-known locking behavior in the important boundary situations. ... I think a lot of stuff there is bit locks. ...
    (Linux-Kernel)
  • Re: Btrfs for mainline
    ... So I don't really know if -rt justifies adaptive locks in mainline/btrfs. ... The spinning does make a big ... In general, we want to spin as long as the lock holder is doing btree searches, ...
    (Linux-Kernel)
  • Re: [PATCH] OsdSynch.c modernization
    ... Spin mutex is too restrictive (e.g., it cannot be used with other ... disabling interrupts while you block on other locks is just as ... If the malloc is using M_NOWAIT you will ...
    (freebsd-current)
  • Re: all mutexes -> read-write locks?
    ... locking primitives it has come to be that mutexes and exclusively ... acquired reader-writer locks are almost the same in terms of overhead ... by all means change it to use rwlocks. ... changing it would allow a slow transition to using rw locks. ...
    (freebsd-arch)