RE: %cpu in system - squid performance in FreeBSD 5.3

From: Jeff Behl (jbehl_at_fastclick.com)
Date: 12/23/04

  • Next message: Bruce A. Mah: "Re: Quick question about the tired ipf/ipnat/"dmz"/bridge scenario" 
    Date: Thu, 23 Dec 2004 10:09:14 -0800
To: <freebsd-performance@freebsd.org>, <freebsd-net@freebsd.org>

     
    As a follow up to the below (original message at the very bottom), I
    installed a load balancer in front of the machines which terminates the
    tcp connections from clients and opens up a few, persistent connections
    to each server over which requests are pipelined. In this scenario
    everything is copasetic:

    last pid: 3377; load averages: 0.12, 0.09, 0.08
    up 0+17:24:53 10:02:13
    31 processes: 1 running, 30 sleeping
    CPU states: 5.1% user, 0.0% nice, 1.8% system, 1.2% interrupt, 92.0%
    idle
    Mem: 75M Active, 187M Inact, 168M Wired, 40K Cache, 214M Buf, 1482M Free
    Swap: 4069M Total, 4069M Free

      PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU
    COMMAND
      474 squid 96 0 68276K 62480K select 0 53:38 16.80% 16.80%
    squid
      311 bind 20 0 10628K 6016K kserel 0 12:28 0.00% 0.00%
    named

    It's actually so good that one machine can now handle all traffic
    (around 180 Mb/s) at < %50 cpu utilization. Seems like something in the
    network stack is responsible for the high %system cpu util...

    jeff

    -----Original Message-----
    From: owner-freebsd-performance@freebsd.org
    [mailto:owner-freebsd-performance@freebsd.org] On Behalf Of Jeff Behl
    Sent: Tuesday, December 07, 2004 9:17 AM
    To: Sean Chittenden
    Cc: freebsd-performance@freebsd.org
    Subject: Re: %cpu in system - squid performance in FreeBSD 5.3

    I upgraded to STABLE but most cpu time is still being spent in system.

    This system is doing ~20Mb/s total with all content being grabbed out of
    memory. I see similar results when running MySQL (a lot of time being
    spent in system)

    Any ideas on what updates to be on the lookout for that might help with
    this? Am I right in guessing that this is a SMP issue and doesn't have
    anything to do with AMD architecture?

    thx

    FreeBSD www2 5.3-STABLE FreeBSD 5.3-STABLE #2: Sun Dec 5 21:06:14 PST
    2004 root@www2.cdn.sjc:/usr/obj/usr/src/sys/SMP amd64

    last pid: 15702; load averages: 0.15, 0.31, 0.31 up
    0+19:55:14 09:09:28
    38 processes: 2 running, 36 sleeping
    CPU states: 5.4% user, 0.0% nice, 12.7% system, 3.4% interrupt, 78.4%
    idle
    Mem: 163M Active, 284M Inact, 193M Wired, 72K Cache, 214M Buf, 1245M
    Free
    Swap: 4069M Total, 4069M Free

      PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU
    COMMAND
      486 squid 96 0 79820K 73996K CPU1 1 110:00 15.04% 15.04%
    squid
      480 squid 96 0 75804K 70012K select 0 105:56 14.89% 14.89%
    squid

    Sean Chittenden wrote:

    >> but the % system time can fluctuate up to 60 at times. My question
    >> is if this is about the type of performance I could expect, or if
    >> people have seen better.
    >
    >
    > I don't know about other people, but I suspect you're running into
    > lock contention. Try using a post 5.3 snapshot (something from
    > RELENG_5) since alc@ has set debug.mpsafevm=1, which lets many calls
    > to the VM run without GIANT, which I suspect is your problem and why
    > the system usage is all over the place. -sc
    >

    _______________________________________________
    freebsd-performance@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-performance
    To unsubscribe, send any mail to
    "freebsd-performance-unsubscribe@freebsd.org"

    howdy,

    I've got a dual proc AMD64 (2gHz) FreeBSD 5.3R system running two squid
    processes (to take advantage of both CPUs). Each process is doing
    around 195 req/s, and the total bandwidth is ~40Mb/s (gig nic via bge
    driver). Squid is being used exclusively as a reverse proxy, with all
    content being served out of memory (very little disk activity).

    Top shows:

    CPU states: 16.0% user, 0.0% nice, 42.7% system, 7.6% interrupt, 33.6%
    idle
    Mem: 898M Active, 569M Inact, 179M Wired, 214M Buf, 171M Free
    Swap: 4069M Total, 4069M Free

    PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND
    14598 squid 108 0 463M 459M select 0 39.2H 59.96% 59.96% squid
    14605 squid 105 0 421M 416M CPU0 1 38.4H 49.95% 49.95% squid

    but the % system time can fluctuate up to 60 at times. My question is
    if this is about the type of performance I could expect, or if people
    have seen better. I was expecting to see much better performance,
    seeing how everything is being served out of memory, but maybe I'm
    asking too much? 400 reqs/s from RAM doesn't seem like much. Is this a
    FreeBSD issue (anybody else with similar experience)? A majority of the
    cpu time being spent in system would seem to indictate such. What is
    all the system load? How can i tell?

    Any help/pointers/remarks appreciated

    thanks,
    jeff
    _______________________________________________
    freebsd-net@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-net
    To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"

  • Next message: Bruce A. Mah: "Re: Quick question about the tired ipf/ipnat/"dmz"/bridge scenario"

    Relevant Pages