[HEADS-UP] mbuma is in the tree

From: Bosko Milekic (bmilekic_at_FreeBSD.org)
Date: 05/31/04

  • Next message: Jon Noack: "Re: misc questions / comments / rambelings" 
    Date: Mon, 31 May 2004 14:51:01 -0700
To: freebsd-current@freebsd.org

    (Hello Chris Haalboom? :-))

    Hello,

      In order to avoid having to type everything again, I'll refer
      to the commit log. PLEASE READ IT IN FULL:

    Bring in mbuma to replace mballoc.

    mbuma is an Mbuf & Cluster allocator built on top of a number of
    extensions to the UMA framework, all included herein.

    Extensions to UMA worth noting:
      - Better layering between slab <-> zone caches; introduce
        Keg structure which splits off slab cache away from the
        zone structure and allows multiple zones to be stacked
        on top of a single Keg (single type of slab cache);
        perhaps we should look into defining a subset API on
        top of the Keg for special use by malloc(9),
        for example.
      - UMA_ZONE_REFCNT zones can now be added, and reference
        counters automagically allocated for them within the end
        of the associated slab structures. uma_find_refcnt()
        does a kextract to fetch the slab struct reference from
        the underlying page, and lookup the corresponding refcnt.

    mbuma things worth noting:
      - integrates mbuf & cluster allocations with extended UMA
        and provides caches for commonly-allocated items; defines
        several zones (two primary, one secondary) and two kegs.
      - change up certain code paths that always used to do:
        m_get() + m_clget() to instead just use m_getcl() and
        try to take advantage of the newly defined secondary
        Packet zone.
      - netstat(1) and systat(1) quickly hacked up to do basic
        stat reporting but additional stats work needs to be
        done once some other details within UMA have been taken
        care of and it becomes clearer to how stats will work
        within the modified framework.

    >From the user perspective, one implication is that the
    NMBCLUSTERS compile-time option is no longer used. The
    maximum number of clusters is still capped off according
    to maxusers, but it can be made unlimited by setting
    the kern.ipc.nmbclusters boot-time tunable to zero.
    Work should be done to write an appropriate sysctl
    handler allowing dynamic tuning of kern.ipc.nmbclusters
    at runtime.

    Additional things worth noting/known issues (READ):
       - One report of 'ips' (ServeRAID) driver acting really
         slow in conjunction with mbuma. Need more data.
         Latest report is that ips is equally sucking with
         and without mbuma.
       - Giant leak in NFS code sometimes occurs, can't
         reproduce but currently analyzing; brueffer is
         able to reproduce but THIS IS NOT an mbuma-specific
         problem and currently occurs even WITHOUT mbuma.
       - Issues in network locking: there is at least one
         code path in the rip code where one or more locks
         are acquired and we end up in m_prepend() with
         M_WAITOK, which causes WITNESS to whine from within
         UMA. Current temporary solution: force all UMA
         allocations to be M_NOWAIT from within UMA for now
         to avoid deadlocks unless WITNESS is defined and we
         can determine with certainty that we're not holding
         any locks when we're M_WAITOK.
       - I've seen at least one weird socketbuffer empty-but-
         mbuf-still-attached panic. I don't believe this
         to be related to mbuma but please keep your eyes
         open, turn on debugging, and capture crash dumps.

    This change removes more code than it adds.

    A paper is available detailing the change and considering
    various performance issues, it was presented at BSDCan2004:
    http://www.unixdaemons.com/~bmilekic/netbuf_bmilekic.pdf
    Please read the paper for Future Work and implementation
    details, as well as credits.

    Testing and Debugging:
        rwatson,
        brueffer,
        Ketrien I. Saihr-Kesenchedra,
        ...
    Reviewed by: Lots of people (for different parts)

    SHOULD YOU HAVE ANY ISSUES:

      - Turn on INVARIANTS
      - Turn on WITNESS
      - Send stack trace and if possible capture crash dump
      - Might require further information from you, please provide
        reachable Email address.
      - When you Email me, please include "MBUMA" in the Subject
        line.

    Cheers,
    Bosko

    _______________________________________________
    freebsd-current@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-current
    To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"

  • Next message: Jon Noack: "Re: misc questions / comments / rambelings"