Re: Locking netatm

2006/5/29, Skip Ford <skip.ford@xxxxxxxxxxx>:

[snip]

So my question is, were network interrupts disabled when mucking
with the atm_timeq list because a generated interrupt can modify
structures within the list? This use is probably very
netatm-specific. I'm still studying the timeout code to
understand what it's doing.

This is the whole point of locking primitives: atomic operations.
Interrupting the code while getting a lock could result in a race condition.

If you have to protect datas through a TSL (Test and Set Lock)
operation, what happen is something like (hypotetically):

if (lock == 0)
lock = 1;
else
thread_sleep();

Than you immagine threadA verify comparisons as true, get interrupted
by interrupt timer and threadB is scheduled to run. threadB executes
the whole procedure (setting lock = 1) than threadA is scheduled again
to run and thinks the critical section is free (since its comparison
was true) so sets again lock=1 and go on. At this point you have an
incorrect behaviour for your OS (a race condition). This is why
interrupts are painful for atomic operations and why they're never
allowed.

A second situation where network interrupts were disabled was for
netatm memory allocation for devices:

in atm_dev_alloc()

s = splimp();

FOREACH(atm_mem_head)
...
malloc (...)

(void) splx(s);

and in atm_dev_free()

s = splimp();

FOREACH(atm_mem_head)
...
free (...);

(void) splx(s);

I'm not sure how these should be protected. Presumably, we don't
want to receive interrupts until the netatm memory for the
device is allocated. Would a global subsystem lock protect these
calls? I can protect atm_mem_head, so maybe that'd be enough?

The better way to protect a list is a rwlock.
Even if they're still incomplete, actually, they're the only primitive
which can assure a two-levels protection and a sort of priority
propagation, so it would be preferable using it (holding according to
your operations).

I think that a lot of queues that we have protected by a sx lock might
switch to rwlocks...

Another use is to protect calls to other subsystems. For
example:

within atm_nif_attach(struct atm_nif *nip)

ifp = nip->nif_ifp;

s = splimp();

if_attach(ifp);
bpfattach(ifp, DLT_ATM_CLIP, T_ATM_LLC_MAX_LEN);

(void) splx(s);
}

and within atm_nif_detach(struct atm_nif *nip)

ifp = nip->nif_ifp;

s = splimp();

bpfdetach(ifp);
if_detach(ifp);
if_free(ifp);

(void) splx(s);

Holding a new netatm subsystem lock won't protect those calls so
I'm not sure how to handle those. Other non-netatm code in the
tree seems to not do any locking at all around those calls.

Here a mutex(9) is enough.

These are really the only uses I've yet to convert so if someone
can provide some pointers, I'd appreciate it. I'm pretty new to
FreeBSD locking, either the old way or the new way. I'm still
studying the code, including other network stacks and the netatm
stack itself, but a pointer or two would be appreciated. I feel
like it's mostly converted, though I've done no testing at all
yet. Once I finish removing splimp(), I can test with the single
subsystem lock, then move on to finer-grained locking where
necessary.

There is not a lot of update documentation about it, I can say you in
particular:
http://www.freebsd.org/doc/en/books/arch-handbook/locking.html
http://www.lemis.com/grog/SMPng/Singapore/paper.pdf
http://people.freebsd.org/~fsmp/SMP/SMP.html

BTW, some of this is not currently applied into the kernel so look at
the code as usual :P

Attilio

--
Peace can only be achieved by understanding - A. Einstein
_______________________________________________
freebsd-arch@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: LinuxPPS & spinlocks
    ... does nothing (if we are already in an interrupt handler). ... context, but doing like this I will disable interrupts ... protect syscalls from each others and from pps_register/unregister, ... Note that *data* is protected by a lock, ...
    (Linux-Kernel)
  • Re: LinuxPPS & spinlocks
    ... does nothing (if we are already in an interrupt handler). ... context, but doing like this I will disable interrupts and I don't ... protect syscalls from each others and from pps_register/unregister, ... Note that *data* is protected by a lock, ...
    (Linux-Kernel)
  • Re: Hello All
    ... section to be used as INTERRUPT PROCEDURE. ... You would normally use more than one INTERRUPT PROCEDURE when you have more than one event that you want to trigger a software interrupt. ... The requirement to lock master records before detail records is controlled by the LOCK TO MODIFY DETAILS option in DASDL. ... It is specified on the master data set. ...
    (comp.sys.unisys)
  • Re: race on multi-processor solaris
    ... > want to block if the lock holder is not running. ... and there is a CPU structure for each CPU. ... interrupts") are handled by "interrupt threads", ... Before we set the waiters bit, we grab the lock protecting the lock's ...
    (comp.unix.solaris)
  • Re: [RFC PATCH] Fair low-latency rwlock v5
    ... If the lock is uncontended, but is not in the current CPU's caches, ... Fair low-latency rwlock provides fairness for writers against reader threads, ... but lets softirq and irq readers in even when there are subscribed writers ... Added contention delays performance tests for thread and interrupt contexts to ...
    (Linux-Kernel)
Loading