cvs-src summary for June 7-14

From: Mark Johnston (mjohnston_at_skyweb.ca)
Date: 06/15/04

  • Next message: Randy Bush: "Re: Upgrading perl" 
    To: current@freebsd.org
Date: Mon, 14 Jun 2004 22:38:09 -0500

    Here's the summary - hope you like locking. I am sorely tempted to go out and
    write some lockless, race-condition-filled code, just for spite.

    On a somewhat related note, I'm considering moving the discussion summary to
    the format that (IIRC) is used by the Linux kernel traffic newsletter, mostly
    direct quotes with some explanation in between. It feels a bit stupid
    paraphrasing everyone's perfectly good comments (and coming up with 50 ways
    to say "replied") for consistency's sake. If this bothers you, g'ahead and
    let me know.

    FreeBSD cvs-src summary for 07/06/04 to 14/06/04
    ++++++++++++++++++++++++++++++++++++++++++++++++
    This is a regular weekly summary of FreeBSD's cutting-edge development.
    It is intended to help the FreeBSD community keep up with the fast-paced
    work going on in FreeBSD-CURRENT by distilling the deluge of data from
    the CVS mailing list into a (hopefully) easy-to-read newsletter. This
    newsletter is marked up in reStructuredText_, so any odd punctuation
    that you see is likely intended for the reST parser.

    .. _reStructuredText: http://docutils.sourceforge.net/rst.html

    You can get old summaries, and an HTML version of this one, at
    http://www.xl0.org/FreeBSD/. Please send any comments to Mark Johnston
    (mark at xl0.org).

    For Lukasz Dudek and Szymon Roczniak's Polish translations of these
    summaries, which may lag the English ones slightly, please see
    http://mocart.pinco.pl/FreeBSD/.

    .. contents::

    ============
    New features
    ============
    ipfw gains lookup table support
    -------------------------------
    Ruslan Ermilov (ru) added support to ipfw for lookup tables, which allows
    a table of IP addresses or subnets to be created and efficiently used in
    rules. The code was originally contributed by Vsevolod Lobko, and refined
    by Ruslan.

    http://www.freebsd.org/cgi/mid.cgi?200406092010.i59KAcXH025699

    ALTQ framework imported
    -----------------------
    Max Laier (mlaier) imported the ALTQ queueing framework from KAME. ALTQ
    provides advanced QoS and queueing for traffic shaping. It was not
    initially enabled, but a later commit linked it to the build; you will
    need to recompile network modules and userland code that uses the ifnet
    structure. As usual, a full buildworld/installworld is safest. The
    queueing, and individual network modules, have not yet changed.

    http://www.freebsd.org/cgi/mid.cgi?200406120010.i5C0AMcc088560
    http://www.freebsd.org/cgi/mid.cgi?200406131729.i5DHTAsD004548

    GEOM-supporting vinum added
    ---------------------------
    Lukas Ertl (le) committed a first version of Vinum, the FreeBSD volume
    manager, updated to use GEOM, the 5.x disk framework. This will allow
    Vinum to be used in 5.x. He also added gvinum, the userland tool to
    manage the GEOMified vinum.

    http://www.freebsd.org/cgi/mid.cgi?200406122116.i5CLGACG006615
    http://www.freebsd.org/cgi/mid.cgi?200406122122.i5CLMlLC008624

    esp driver added
    ----------------
    Scott Long (scottl) imported the sbus version of the NetBSD esp driver,
    which will handle the SCSI card in Sun Ultra 1e and 2 machines. Once the
    PCI version is ported, the driver can also replace the current amd driver,
    which handles amd53c974-based SCSI cards.

    http://www.freebsd.org/cgi/mid.cgi?200406100511.i5A5Beog063878

    IP over Firewire support added
    ------------------------------
    Doug Rabson (dfr) added a new driver to support IP over Firewire. The
    new driver conforms to `RFC 2734`_ and `RFC 3146`_, and is intended to
    eventually supersede the fwe driver.

    .. _`RFC 2734`: ftp://ftp.rfc-editor.org/in-notes/rfc2734.txt
    .. _`RFC 3146`: ftp://ftp.rfc-editor.org/in-notes/rfc3146.txt

    http://www.freebsd.org/cgi/mid.cgi?200406131054.i5DAsbKU005744

    DRM code updated
    ----------------
    Eric Anholt (anholt) updated the DRM (Direct Rendering Manager) code from
    DRI (Direct Rendering Infrastructure) CVS. The update consists mostly of
    new PCI IDs and a new packet for the Radeon card.

    http://www.freebsd.org/cgi/mid.cgi?200406110327.i5B3R0nc088530

    sha1 and rmd160 checksum utilities added
    ----------------------------------------
    Oliver Eikemeier (eik) added sha1 and rmd160 checksum-calculating tools,
    which are similar to the existing md5 tool and used in the same way.

    http://www.freebsd.org/cgi/mid.cgi?200406111607.i5BG72iv068697

    Locking added to UNIX domain sockets, among other things
    --------------------------------------------------------
    Robert Watson (rwatson) added locking to UNIX domain sockets. This code
    was submitted by Sam Leffler (sam), and was originally written for BSD/OS,
    with Robert doing several tweaks to it. He also made small locking
    commits to a number of other subsystems, mostly networking-related.

    http://www.freebsd.org/cgi/mid.cgi?200406102134.i5ALYcNr004704
    (many others)

    CVS updated to 1.11.17
    ----------------------
    Peter Wemm (peter) updated the system CVS to version 1.11.17. This
    replaces the old version 1.11.15. 1.11.17 is a security and bugfix
    release, although the security updates were previously applied.

    http://www.freebsd.org/cgi/mid.cgi?200406101912.i5AJCpV5066737

    ===============
    Notable changes
    ===============
    COMPAT_SUNOS removed
    --------------------
    Poul-Henning Kamp (phk) eliminated COMPAT_SUNOS, compatibility code for
    SunOS 4.x, which came from BSD4.4-Lite1's sparc32 port.

    http://www.freebsd.org/cgi/mid.cgi?200406111116.i5BBGRVA098172

    struct tty now reference counted
    --------------------------------
    Poul-Henning Kamp (phk) committed reference counting for the tty code;
    however, drivers are not yet using the new calls.

    http://www.freebsd.org/cgi/mid.cgi?200406090941.i599fUbQ068884

    =================
    Discussion topics
    =================
    A few words on locking
    ----------------------
    Poul-Henning Kamp (phk) fixed a race condition in the process management
    code, prompting a huge (but productive) thread about locking. I have
    summarized most of it, but left out some subthreads that were essentially
    duplicates of others.

    Initially, Poul-Henning posted with a description of why the old locking
    was wrong - it locked an object, decremented its reference count, then
    unlocked it and took the reference count again to decide whether to
    destroy it. If another thread decremented the reference count and
    destroyed the object in the meantime, the first thread will try to take
    the reference count of the dead object and crash. His suggestion was to
    save the reference count and use that when deciding whether to destroy the
    object, rather than trying to take it again.

    Nate Lawson (njl) noted that if another thread increments the reference
    count after the unlock but before the destruction, you could destroy an
    object that was still in use. He suggested that you could hold the lock
    until the object is freed. Brian Feldman (green) suggested locking the
    object as well as the list of objects of that type, so once the reference
    count reaches zero, it can't be found to be incremented again. Darren
    Reed (darrenr) agreed, and suggested an alternate way, using separate read
    and write locking to improve performance.

    Lukas Ertl (le) had a similar followup to Nate's, asking whether it's
    possible for another thread to increment the reference count after it's
    found to be zero. Garrett Wollman (wollman) answered that that would only
    be possible if not all the references were counted. Ruslan Ermilov (ru)
    asked him to clarify whether he meant that a change from 0 to 1 wasn't
    possible; Julian Elischer (julian) responded, answering that once the
    object is removed from the list, and only one reference (the one in your
    own code) remains, nobody else can find the object to increment it.

    Bosko Milekic (bosko) responded to a request by M. Warner Losh (warner)
    for clarification, explaining how the race condition works. He
    also posted code for preventing a similar problem when using an
    atomically-updated reference counter, instead of one protected by a lock.
    Warner suggested that a locking section in the handbook would be a good
    place for the techniques discussed. Nate Lawson followed up that Bosko's
    new code could have a race if the reference count increment path isn't
    done properly. Bosko responded that it could not, since the reference
    count is done atomically and it can't go up once it reaches zero, since
    the code that set it to 0 had the last reference.

    Pawel Jakub Dawidek (pjd) followed up to Bosko's post about atomic
    reference counts, wondering whether it's limited to values of 0 and 1.
    Bosko replied, saying that it can count any number of references. Pawel
    asked whether atomic operations are also atomic on MP machines; Warner
    answered that they are, which makes them very bad for performance.
    Poul-Henning also replied, agreeing that atomic operations are not cheap
    and mutexes should be generally used instead, since they are easier to
    understand and validate with WITNESS. Julian also warned that the atomic
    method could result in a page fault.

    Julian suggested that a more formal way of handling reference counts is
    needed; Poul-Henning replied that it is often bad for performance and code
    clarity. Julian argued that most people found it worth pursuing anyway;
    Poul-Henning encouraged him to go ahead with it. Julian offered to take
    discussion of such a formal system to -arch, if anyone was interested.

    Warner responded to Julian's post, saying that documentation of how to
    properly do reference counting is needed. Poul-Henning suggested that
    people should be familiar with locking from computer science courses;
    Warner suggested documentation would still help, and Poul-Henning agreed,
    saying it was on his list of articles to write in the future.

    Mike Silbersack (silby) chimed in, titling the thread "Locking the
    Bikeshed". I decided to keep my title, though, simply out of fear of
    wearing out my k key.

    http://www.freebsd.org/cgi/mid.cgi?200406090929.i599T8h6065944

    ===================
    Important bug fixes
    ===================
    3ware 6xxx RAID bug fixed
    -------------------------
    Vinod Kashyap (vkashyap) patched a bug in the twe driver for 3ware RAID
    controllers that was causing 6xxx series controllers to resubmit the same
    request repeatedly when the command queue was full. He also merged the
    fix to 4.x.

    http://www.freebsd.org/cgi/mid.cgi?200406111857.i5BIvWCd012558

    ===============
    Other bug fixes
    ===============
    Christian S. J. Peron (csjp) patched the ipfw code so that PCBs will be
    looked up only once per ipfw activation. This should significantly speed
    up UID/GID-based firewall rules. He provided a `benchmark graph`_ showing
    the speedup.

    .. _`benchmark graph`: http://people.freebsd.org/~csjp/ip_fw_pcb.png

    http://www.freebsd.org/cgi/mid.cgi?200406112217.i5BMHFfM064236

    David Schultz fixed an error in the code for rounding floating-point
    numbers to integers that resulted in incorrect values for some inputs.

    http://www.freebsd.org/cgi/mid.cgi?200406092124.i59LOqsL049229

    Ian Dowse (iedowse) fixed a bug in the USB code that was causing a
    "restarting port 1" error followed by "giving up port 1" on a VIA USB 1
    controller. This fix was also MFC'ed.

    http://www.freebsd.org/cgi/mid.cgi?200406100113.i5A1DRvB006038

    Ruslan Ermilov (ru) fixed a crash in make on 4.x when a .DEFAULT target in
    a Makefile has no commands.

    http://www.freebsd.org/cgi/mid.cgi?200406101307.i5AD7rYM080168
    _______________________________________________
    freebsd-current@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-current
    To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"

  • Next message: Randy Bush: "Re: Upgrading perl"

    Relevant Pages

    • Re: Threads - why isnt a whole object locked when ...?
      ... All of your questions are answerable by pointing out that you have an incorrect mental model of what it means to "lock" something. ... That is, while it's entirely sensible to people who are familiar with standard thread synchronization techniques, in reality it's not actually locking anything. ... Instead, it's using the reference as a sort of traffic signal for other threads, which are cooperating, to respect. ...
      (microsoft.public.dotnet.languages.csharp)
    • Re: multi threading in C#
      ... guarantees that no-one else can screw things up by locking on it ... reference example, "listLock" is the reference only you have access to, and the implication is that "queue" is a reference that others have access to. ...
      (microsoft.public.dotnet.languages.csharp)
    • Re: [PATCH] reiserfs: Expand i_mutex to enclose lookup_one_len
      ... version of the code where I kept a reference to /.reiserfs_priv/xattrs. ... Then I decided that .reiserfs_priv was all I needed to cache (to avoid ... recursive i_mutex locking on the fs root) and dropped the caching of ...
      (Linux-Kernel)
    • Re: [RFC] ext3/jbd race: releasing in-use journal_heads
      ... reference in b_jcount. ... Now if you unfile a buffer from the list you 'get back' it's ... the proper list locking so that two different callers cannot 'get back' one ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • SUMMARY : trouble with fsck
      ... ** Phase 4 - Check Reference Counts ... ** Phase 5 - Check Cylinder Groups ... 0.3% fragmentation) ... CORRECT BAD CG SUMMARIES? ...
      (SunManagers)