Re: Automated performance testing

From: Eric Anderson (anderson_at_centtech.com)
Date: 01/31/05

  • Next message: Arne : "Re: Automated performance testing" 
    Date: Mon, 31 Jan 2005 09:45:34 -0600
To: Robert Watson <rwatson@freebsd.org>

    Robert Watson wrote:
    > On Sun, 30 Jan 2005, Jim C. Nasby wrote:
    >
    >
    >>With all the discussion of performance testing between 4.11, 5.3, and
    >>Linux, would it be useful to make performance testing part of the
    >>automated testing that already occurs (via tinderbox, iirc). Doing so
    >>might make it easier to detect performance impacting changes, as well as
    >>making performance testing easier in general.
    >
    >
    > Yes, it would be quite valuable. I've been hoping to set up something
    > like this for a while, but have never found the opportunity. I have been
    > tracking the long term behavior of MySQL performance as part of the
    > netperf work, but because testing is fairly hardware and time consuming,
    > the polling intervals are uneven, and not quite close enough to nail down
    > culprits. I'd really like to see a small and fairly well-defined set of
    > tests run every couple of days so we can show long term graphs, and catch
    > regressions quickly. Unfortunately, this is a bit harder than
    > tinder-boxing, because it involves swapping out whole system
    > configurations, recovering from the inevitable failure modes, etc, which
    > proves to be the usual sticking point in implementing this. However, I'd
    > love to see someone work on it :-).

    Maybe it would help to come up with a list of tests, or even just things to be tested initially. Then one could get a sense of what tests need to be run, and how to do it.. I wonder if it's worth building a little app that runs a suite of other included apps (found in the ports collection), grabs system hardware and configuration information, runs the tests, and sends them to a central location. Then we can have OS, hardware, software config, etc information, and compare system vs system, OS ver, etc, and people just build the fbsdperftest port (which installs a slew of other ports, like iozone, bonnie, apache (HTTP testing), mysql (DB testing?), etc), then they run the program, it gathers the data, asks a couple of questions, and sends the data off to a central spot for analyzing later. Then, one could set up a machine that just does the cvsup, buildworld, ..., fbsdperftest, repeat..

    Eric 

    -- 
------------------------------------------------------------------------
Eric Anderson        Sr. Systems Administrator        Centaur Technology
I have seen the future and it is just like the present, only longer.
------------------------------------------------------------------------
_______________________________________________
freebsd-performance@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org"
  • Next message: Arne : "Re: Automated performance testing"

    Relevant Pages

    • Re: Automated performance testing
      ... > automated testing that already occurs (via tinderbox, ... > making performance testing easier in general. ... tracking the long term behavior of MySQL performance as part of the ...
      (freebsd-performance)
    • Automated performance testing
      ... Linux, would it be useful to make performance testing part of the ... automated testing that already occurs ... making performance testing easier in general. ...
      (freebsd-performance)