Re: Request for Comments: libarchive, bsdtar

From: Tim Kientzle (kientzle_at_acm.org)
Date: 01/15/04

  • Next message: David Schultz: "Re: Request for Comments: libarchive, bsdtar" 
    Date: Thu, 15 Jan 2004 13:28:31 -0800
To: David Schultz <das@FreeBSD.ORG>

    David Schultz wrote:
    >
    > ... the package tools don't have an archive format
    > that supports random access, so they wind up extracting a
    > temporary copy of the whole archive, then processing the parts
    > that they want.

    <grin> Funny you should mention this.

    As it turns out, random access is not necessary to avoid the
    temporary copy. To prove it, I've written a pkg_add replacement
    that extracts directly to the final location in a single
    pass with no temporary directory and no need for random
    access to the package file. In particular, it can extract
    and install a package directly from stdin or an FTP site
    with no disk overhead at all.

    How? Easy. Use libarchive to extract the packing
    list---which is always the first archive entry---directly
    into memory. Parse the packing list to find out where
    everything goes, then extract files from the package
    directly to their final location. Anything that needs to be
    ignored just gets skipped over. No random access necessary.

    My prototype pkg_add works and it's about three times faster than
    the pkg_add in the tree. However, it's also a bit of a mess.
    libarchive is a big step towards cleaning up that mess by
    separating out the tarfile reading code. My next project is to
    separate out some of the generic package management
    functions into a libpkg library. At that point, I should
    be able to overhaul my prototype to use these libraries
    and finally have something production-quality.

    It is certainly possible that cleaner handling of package
    files might allow us to eventually use package-management
    tools for the sysinstall distribution files, but that's
    certainly not a short-term concern.

    My goal right now is just to clean up the existing package tools.
    In particular, my new package tools will:
       * have the same names as the current tools
       * use the same command-line options (as far as possible <sigh>)
       * use the exact same file format as the current tools
       * be built from general-purpose modular library components

    This last item is key. This will make it easier to maintain
    and improve these tools over time with new capabilities
    (better conflict management, for example) and even new kinds
    of package tools.

    As I said, libarchive is just the first major product of
    this program. There's still a lot of work to do, but I
    have a roadmap and I'll keep chugging down it as my limited
    time allows.

    Tim

    _______________________________________________
    freebsd-arch@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-arch
    To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"

  • Next message: David Schultz: "Re: Request for Comments: libarchive, bsdtar"

    Relevant Pages