Re: speed of bzip2 versus gzip

On Fri, Jul 20, 2007 at 05:50:20PM -0700, Chuck Swiger wrote:
On Jul 20, 2007, at 5:37 PM, Norberto Meijome wrote:
Is it normal for bzip2 to be significantly slower than gzip?
If not, where can I look for things that might be causing
"bzip2 --fast" to take 50-60 times longer to compress a
(sendmail log) file than gzip?

i never measured it to see if it is 50-60 times slower, but yes, gzip
blows
bzip2 out of the water on speed. I wanted to use bzip2 to compress
multi-GB
weblog files, but gzip beat it my miles, and bzip2 wasn't THAT much better
@
compressing it to make it worth it.

Thanks for the feedback, Norberto.

Of course, it all depends on what your priorities are, too-- if what you
want is a final tarball which is being mirrored and downloaded frequently,
then your goal is to obtain the absolute best compression, and how much CPU
--best takes isn't important.

Comparing the default (-5 compression?) of gzip to bzip2 would probably be
more reasonable if you care about reasonably timely compression.

If I read the man page correctly, bzip2 defaults to --best, which is why
I compared gzip to bzip2 --fast. With the 1.5G sendmail log, bzip2 --fast
compresses to just under 10M in about 55 minutes, give or take. bzip2
--best compresses 1.5G to 1.8M, but takes about 2.25 hours. gzip
compresses almost as well (with 3% or so) as --fast, but does it in 1
minute instead of 55 on a dual P-III 1.4GHz (but of course, using only
one CPU).

Jim
_______________________________________________
freebsd-questions@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: speed of bzip2 versus gzip
    ... i never measured it to see if it is 50-60 times slower, but yes, gzip ... I wanted to use bzip2 to compress ... then your goal is to obtain the absolute best compression, ... sys 0m0.257s ...
    (freebsd-questions)
  • Re: Linux Kernel Source Compression
    ... gzip is the winner again. ... usually two to six times more time than lzma, that is, four to twelve ... decompress the faster the smaller the compressed size is, while bzip2 ... gets slower when the compression ratio gets better. ...
    (Linux-Kernel)
  • Re: emulate an end-of-media
    ... the compression ratio on a dump of my home directory ... bzip2 took about 74min to compress, gzip only 11minutes. ...
    (freebsd-hackers)
  • Re: speed of bzip2 versus gzip
    ... file than gzip? ... I wanted to use bzip2 to compress multi-GB ... Of course, it all depends on what your priorities are, too-- if what you want is a final tarball which is being mirrored and downloaded frequently, then your goal is to obtain the absolute best compression, and how much CPU --best takes isn't important. ... Comparing the default of gzip to bzip2 would probably be more reasonable if you care about reasonably timely compression. ...
    (freebsd-questions)
  • Re: Transparent compression in the FS
    ... >>Note that a file compressed with bzip2 is not necessarily smaller than ... >>the same file compressed with gzip. ... > Have you noticed that with real-life data, ... whereas bzip2 is better at binary compression. ...
    (Linux-Kernel)