System hang / extreme slowdown under MySQL

From: Murray Taylor (MTaylor_at_bytecraft.com.au)
Date: 09/29/05

  • Next message: Daniel O'Connor: "Re: FreeBSD 5.4 AMD64 & MPD"
    Date: Thu, 29 Sep 2005 09:28:28 +1000
    To: <freebsd-hackers@freebsd.org>
    
    

    We are using MySQL on a FreeBSD box for one of our systems
    and recently after doing a number of application code changes to
    support a new client the system experiecned what seemed to be
    a total hang. It didnt crash per se, no log messages
    were caught.... it just went mostly unresponsive.

    The data base is accessed via a set of TCL, C and web apps.

    The symptoms are no response to any of the programs, yet
    things like switching virtual consoles etc work ok. But it is
    not possible to login... one can type 'root', see the characters echoed,
    press enter and get nothing..

    If there is an open ssh session or open console session, one
    could issue a single command, like 'ls', get the response, then say
    issue a 'ps' and only get the header line.....

    On one of the occasions as we fly in to reboot the box, one of the
    users said that it _was working_ just V E R Y S L O W L Y....
    like 3 web pages in 30 minutes.

    We did find some unclosed database queries in the TCL code that
    have been fixed, but based on a sysctl that 'seems to be significant'
    we may not yet have found all our problems.

    The sysctl that we have picked is the one printed below and in
    particular
    the crosspoint PV ENTRY: / USED value. this is slowly but consistently
    increasing
    on this box, yet remains stable on all other FreeBSD boxes.

    THE QUESTIONs are:
    Is this the correct sysctl to be monitoring?
    Is there a good description of the sysctl identified re relevance
    of the numbers to the actual system condition?

    Does anyone have any other ideas of things to look for that may
    causing the hangs?

    The system is
    FreeBSD svmysql1.dand07.au.bytecraft.au.com 4.10-RELEASE-p7 FreeBSD
    4.10-RELEASE-p7 #3: Thu Apr 14 15:34:37 EST 2005

    Both the old system (Mysql 3.23) and the current one
    (mysql Ver 12.22 Distrib 4.0.22, for portbld-freebsd4.10 (i386))
    are doing this.

    Tcl version is 8.4

    vm.zone:
    ITEM SIZE LIMIT USED FREE REQUESTS

    PIPE: 160, 0, 13, 89, 2208898
    SWAPMETA: 160, 233016, 0, 0, 0
    unpcb: 160, 0, 13, 62, 18543
    ripcb: 192, 12328, 0, 42, 5
    syncache: 160, 15359, 0, 51, 1342401
    tcpcb: 576, 12328, 539, 3059, 1341005
    udpcb: 192, 12328, 3, 39, 9904
    socket: 224, 12328, 556, 3064, 1369458
    DIRHASH: 1024, 0, 652, 4, 767
    KNOTE: 64, 0, 1, 127, 7589
    VNODE: 192, 0, 33713, 101, 33713
    NAMEI: 1024, 0, 0, 16, 11767321
    VMSPACE: 192, 0, 39, 89, 585870
    PROC: 416, 0, 72, 75, 585929
    DP fakepg: 64, 0, 0, 0, 0
    PV ENTRY: 28, 2281256, 78652, 445576, 435851412
    MAP ENTRY: 48, 0, 1084, 829, 40997106
    KMAP ENTRY: 48, 57423, 265, 119, 2234109
    MAP: 108, 0, 7, 3, 7
    VM OBJECT: 92, 0, 26533, 211, 15431058

    -- 
    A: Because it fouls the order in which people normally read text.
    Q: Why is top-posting such a bad thing?
    A: Top-posting.
    Q: What is the most annoying thing on usenet and in e-mail?
    Murray Taylor
    Bytecraft Systems
    P: +61 3 8710 2555
    F: +61 3 8710 2599
    D: +61 3 9238 4275
    E: mtaylor@bytecraft.com.au
    ---------------------------------------------------------------
    The information transmitted in this e-mail is for the exclusive
    use of the intended addressee and may contain confidential
    and/or privileged material. Any review, re-transmission,
    dissemination or other use of it, or the taking of any action
    in reliance upon this information by persons and/or entities
    other than the intended recipient is prohibited. If you
    received this in error, please inform the sender and/or
    addressee immediately and delete the material. 
    E-mails may not be secure, may contain computer viruses and
    may be corrupted in transmission. Please carefully check this
    e-mail (and any attachment) accordingly. No warranties are
    given and no liability is accepted for any loss or damage
    caused by such matters.
    ---------------------------------------------------------------
    ***This Email has been scanned for Viruses by MailMarshal.***
    _______________________________________________
    freebsd-hackers@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
    To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
    

  • Next message: Daniel O'Connor: "Re: FreeBSD 5.4 AMD64 & MPD"

    Relevant Pages

    • Re: Network Problem in FreeBSD 6.2-RELEASE-p5
      ... > Is the FreeBSD machine performing NAT for the Windows box? ... > extazyti: yes with PF firewall. ... > ports It's not really open because there is no program to open random ... particular sysctl. ...
      (freebsd-questions)
    • Re: regression in HDA functionality
      ... out-of-the-box sound output does not work. ... to, and be changeable in sysctl, defaulting to "auto" or something. ... new users adopting FreeBSD who might one day be a FreeBSD kernel ... for end-users, but may look trivial to devs and thus overlooked. ...
      (freebsd-current)
    • Re: simple network traffic query tool
      ... os i want to write a widget which uses sysctl or any other ... I'm using a modified version of Superkarambas CompactMonitor. ... but easily adaptable for freebsd. ... awk ''" align=right interval=1000 ...
      (freebsd-questions)
    • Re: wierd dsl performance with -CURRENT
      ... Your Macintosh processes network traffic differently than FreeBSD ... dispatch, dealing with polling, tuning TCP paramters via sysctl, ... -performance or -net are much better lists to use than -current, ...
      (freebsd-current)
    • Re: MySQL Performance 6.0rc1
      ... Nevertheless I run a production MySQL on a double-Xeon (Supermicro ... etc) which eventually kill MySQL with FreeBSD ... CPU: AMD OpteronProcessor 246 ... <ACPI PCI bus> on pcib0 ...
      (freebsd-current)