Re: Request for Comments: libarchive, bsdtar

From: Tim Kientzle (kientzle_at_acm.org)
Date: 01/15/04

  • Next message: Don Lewis: "Re: Request for Comments: libarchive, bsdtar" 
    Date: Wed, 14 Jan 2004 19:39:51 -0800
To: Vladimir Dozen <vladimir-dozen@mail.ru>

    Vladimir Dozen wrote:
    > > so I want to make sure it's well-tested
    >
    > BTW, how do you perform testing of the library/tar?

    I've used automated tests to verify a few of the trickier
    routines, such as exercising boundary conditions in the
    formatting and parsing logic.

    There are a number of built-in logic tests in the code. Most
    notably, each public function starts with a call to
    "archive_check_magic" which verifies that the provided
    archive structure is in the correct state.

    I've also been collecting sample test archives to verify
    correct operation. Joerg Schilling's collection of test
    files has been very helpful.

    I haven't yet had a chance to automate the full-program
    tests, though. I have a few ideas about how to proceed
    and what needs testing, but haven't yet pieced
    anything together.

    I also plan to use dmalloc (or something similar) to
    test for memory leaks and invalid heap operations.
    I used some crude, home-grown routines early in development
    to verify memory usage but haven't had a chance to do
    more systematic testing.

    I have tested enough to know that:
       * libarchive correctly archives 64k pathnames
       * performance is comparable to gtar overall
       * When reading/writing compressed archives,
         zlib/bzlib are the performance bottlenecks.
         bzlib, in particular, seems very sensitive to
         the size of blocks you feed it; some work to
         compress/decompress larger blocks would be helpful
       * When writing non-compressed archives, getpwent/getgrent
         calls in bsdtar are the most obvious performance issue
         (about 10% of the CPU time for tar -cf /dev/null)
         I'm considering a simple LRU cache of uname/gname
         lookups to address this.

    > >> - I would prefer it if compression was done by opening a pipe to
    > >> gzip/bzip2 instead of using libz/libbz2.
    > ...
    > The right way to avoid code duplication between gzip and libarchive
    > is to use common libgzip. The same applies to bzip and compress.

    Yes, I use zlib and bzlib to handle compression. This handles
    essentially everything except for the relatively minor task of
    generating/verifying the gzip header (newer versions of zlib
    handle even that, but the version we currently have in the tree
    does not).

    I would love to see a library version of compress(1) with
    an API similar to that of zlib/bzlib.

    Tim

    _______________________________________________
    freebsd-arch@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-arch
    To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"

  • Next message: Don Lewis: "Re: Request for Comments: libarchive, bsdtar"

    Relevant Pages