serious slowdown in 5.3-BETA1 using SCHED_4BSD and SMP

From: Oliver Lehmann (oliver_at_FreeBSD.org)
Date: 08/29/04

  • Next message: Geert Hendrickx: "spreading partitions over multiple drivers"
    Date: Sun, 29 Aug 2004 15:59:45 +0200
    To: re@freebsd.org
    
    
    

    Hello,

    I've noticed a serious speeddown by using SMP. When I run the same system
    with only one CPU, the system works quite well. But after sticking in the
    second CPU, the system slows down noticeable. The whole system responds
    quite laggy.
    That started to happen after I updated FreeBSD from 5.2.1 to 5.3-BETA1. On
    5.2.1 I had all kernel debugging options enabled. With that kernel-config
    5.3 was not useable in any way. For example playing 128kbit mp3s over nfs
    was nearly impossible because it stoped playing every 10-30 seconds for
    2-5 seconds....
    I did a "ln -s aj /etc/malloc.conf" and disabled all the [DGK]DB, WITNESS*
    and INVARIANT* options in my Kernel-config file. IT tokk me 9:52h(!!) to
    compile that kernel (make kernel).

    Now, booted with that kernel (NUDEL.bak) - SMP feels still much more
    slower then 5.2.1 felt.

    To show the speeddowns, I built now the attached kernel "NUDEL":

    ##########################################################################
    #### on System: nudel.dmesg
      Booted kernel: "NUDEL.bak" (SCHED_4BSD)
      1 CPU : 3927.42 real 3279.99 user 454.95 sys
      2 CPUs: 51546.74 real 44414.65 user 5151.46 sys (14hrs!!!)

      Booted kernel: "NUDEL" (SCHED_ULE)
      1 CPU : 3943.72 real 3284.04 user 459.58 sys
      2 CPUs: 3864.97 real 3287.69 user 454.52 sys

    ##########################################################################
    #### on System: curry.dmesg
      Booted kernel: "CURRY" (SCHED_4BSD)
      1 CPU : 4376.41 real 3652.60 user 435.60 sys

    I built with:

    rm -rf /usr/obj/i386-5.3/usr/src/sys/NUDEL && \
    /usr/bin/time env MAKEOBJDIRPREFIX=/usr/obj/i386-5.3 make kernel

    My make.conf:

    KERNCONF?=NUDEL
    # MAKEOBJDIRPREFIX=/usr/obj/i386-5.2.1
    WRKDIRPREFIX=/usr/obj/${MACHINE}-${OSREL}
    #CPUTYPE=p2
    # -- use.perl generated deltas -- #
    # Created: Fri Mar 21 00:28:26 2003
    # Setting to use base perl from ports:
    PERL_VER=5.6.1
    PERL_VERSION=5.6.1
    PERL_ARCH=mach
    NOPERL=yo
    NO_PERL=yo
    NO_PERL_WRAPPER=yo

    I've attached the vmstat -i output during make

    For example - a simple "make buildworld" with 2CPUs/SCHED_4BSD for (and
    on) 5.2.1-RC2 took only 16916.70sec (I'm sorry, but I've no kernel
    building times archived)

    !!!And it's not just the kernel build - the whole system with running
    2 CPUS + SCHED_4BSD is _slow_. I just took the kernel build as an
    example!!!

    Something seems to be verry broken with SCHED_4BSD+SMP

    -- 
     Oliver Lehmann
      http://www.pofo.de/
      http://wishlist.ans-netz.de/
    -- 
     Oliver Lehmann
      http://www.pofo.de/
      http://wishlist.ans-netz.de/
    
    
    
    
    
    
    
    
    
    
    

    _______________________________________________
    freebsd-performance@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-performance
    To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org"











  • Next message: Geert Hendrickx: "spreading partitions over multiple drivers"

    Relevant Pages

    • Re: SMP detection
      ... I have the SMP-GENERIC kernel ... I've decided to disable SMP from BIOS. ... way to install one already precompiled? ... cpu and the system monitor only features one ...
      (freebsd-questions)
    • different oops on xeon with smp kernel
      ... then SMP kernels oops very early at boot at different places. ... CPU: Trace cache: 12K uops, ... Unable to handle kernel paging request at virtual address 8528244c ...
      (Linux-Kernel)
    • Oops with 2.6.1_rc1-mm1
      ... Unable to handle kernel NULL pointer dereference at virtual address ... CPU: 0 ... EIP is at vt_ioctl+0x1e/0x1f00 ... Pin 2-17 already programmed ...
      (Linux-Kernel)
    • [OOPS] hotplugging cpus via /sys/devices/system/cpu/
      ... then the CPU doesn't start and the kernel prints out an oops. ... Using 1 I/O APICs ... Linux Plug and Play Support v0.97 Adam Belay ...
      (Linux-Kernel)
    • Re: 2 x quad-core system is slower that 2 x dual core on FreeBSD
      ... Can you try a generic SMP kernel (called "SMP" in 6.x; ... relationship between how many CPUs are running and what the sys load is. ... Now I'm running yesterday's FreeBSD 7.0-BETA3 amd64 with GENERIC kernel. ... I.e. it seems to work good as a simple server with not so powerfull CPU. ...
      (freebsd-stable)