HEADS UP: Giant-free networking now the default in CVS HEAD (6.x)

From: Robert Watson (rwatson_at_FreeBSD.org)
Date: 08/28/04

  • Next message: Wilko Bulte: "5.3-BETA displaying cd0 as an install target?"
    Date: Sat, 28 Aug 2004 11:20:40 -0400 (EDT)
    To: current@FreeBSD.org
    
    

    HEADS UP!

    The default setting for debug.mpsafenet has changed from 0 to 1, which
    means that by default, a number of kernel elements will now run without
    Giant by default, and often in parallel or preemptively. This includes:

    - Network interface ithreads for supported (MPSAFE) network device
      drivers.

    - The netisr thread when processing MPSAFE network protocols.

    - Many network-related system calls, including the send and receive paths.

    - Many network related timeout and callout routines happening with clock
      ticks.

    As mentioned in the commit message, there are now a number of seatbelts
    and ways to change the setting to restore more conservative operation, as
    this change may expose existing bugs or introduce new bugs. In fact, that
    is highly likely to be the case. If you experience new network related
    problems, as a first debugging step, please try putting the Giant lock
    back over the network stack. This can be done by setting the loader
    tunable "debug.mpsafenet" to 0, or by compiling "options NET_WITH_GIANT"
    into your kernel configuration. If the problem goes away, that doesn't
    necessarily mean it's a bug in MPSAFE operation, as we could just be
    encouraging an existing bug to expose itself, but it's also fairly likely
    it is.

    If you want to run in an environment with even more parallelism, you can
    set net.isr.enable to 1 to run network processing to completion in
    interrupt threads, which has the effect of deferring less work to the
    netisr.

    If you'd like to read more about the on-going work, including seeing
    earlier posts about the work, precautions to take, etc, you can read about
    it at:

        http://www.watson.org/~robert/freebsd/netperf/

    I do anticipate a slightly bumpy ride with this setting now as the
    default, but we have easy ways to back off the change at run-time and
    compile-time should it get too bumpy. There will likely be some serious
    nits we pick up and fix in the next three to four days, in particular, so
    some caution might need to be exercised.

    Robert N M Watson FreeBSD Core Team, TrustedBSD Projects
    robert@fledge.watson.org Principal Research Scientist, McAfee Research

    ---------- Forwarded message ----------
    Date: Sat, 28 Aug 2004 15:11:13 +0000 (UTC)
    From: Robert Watson <rwatson@FreeBSD.org>
    To: src-committers@FreeBSD.org, cvs-src@FreeBSD.org, cvs-all@FreeBSD.org
    Subject: cvs commit: src/sys/conf options src/sys/sys kernel.h src/sys/net netisr.c

    rwatson 2004-08-28 15:11:13 UTC

      FreeBSD src repository

      Modified files:
        sys/conf options
        sys/sys kernel.h
        sys/net netisr.c
      Log:
      Change the default disposition of debug.mpsafenet from 0 to 1, which
      will cause the network stack to operate without the Giant lock by
      default. This change has the potential to improve performance by
      increasing parallelism and decreasing latency in network processing.
      
      Due to the potential exposure of existing or new bugs, the following
      compatibility functionality is maintained:
      
      - It is still possible to disable Giant-free operation by setting
        debug.mpsafenet to 0 in loader.conf.
      
      - Add "options NET_WITH_GIANT", which will restore the default value of
        debug.mpsafenet to 0, and is intended for use on systems compiled with
        known unsafe components, or where a more conservative configuration is
        desired.
      
      - Add a new declaration, NET_NEEDS_GIANT("componentname"), which permits
        kernel components to declare dependence on Giant over the network
        stack. If the declaration is made by a preloaded module or a compiled
        in component, the disposition of debug.mpsafenet will be set to 0 and
        a warning concerning performance degraded operation printed to the
        console. If it is declared by a loadable kernel module after boot, a
        warning is displayed but the disposition cannot be changed. This is
        implemented by defining a new SYSINIT() value, SI_SUB_SETTINGS, which
        is intended for the processing of configuration choices after tunables
        are read in and the console is available to generate errors, but
        before much else gets going.
      
      This compatibility behavior will go away when we've finished the last
      of the locking work and are confident that operation is correct.
      
      Revision Changes Path
      1.478 +1 -0 src/sys/conf/options
      1.12 +84 -3 src/sys/net/netisr.c
      1.118 +6 -0 src/sys/sys/kernel.h

    _______________________________________________
    freebsd-current@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-current
    To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"


  • Next message: Wilko Bulte: "5.3-BETA displaying cd0 as an install target?"